Community discovery method based on attention fusion and neighborhood information enhancement

Abstract

The invention relates to a community discovery method based on attention fusion and neighborhood information enhancement, and belongs to the technical field of community discovery. The method comprises the steps of preparing original data of a graph, obtaining enhanced neighborhood structure information representation of an original graph structure, and meanwhile obtaining attribute information representation of the original graph through an auto-encoder; an attention mechanism is adopted to fuse enhanced neighborhood structure information representation and attribute information representation, and an image clustering result is obtained by calculating soft allocation; and constructing a supervision loss function, and optimizing the graph clustering result through self-supervision training. According to the method, high-order neighborhood information and a graph attention mechanism are introduced, so that the utilization effect of the neighborhood information is enhanced; meanwhile, the importance of different neighbor nodes is fully considered, finer topological structure information is obtained, and the extracted enhanced neighborhood topological structure information and attribute information are dynamically fused by using an attention mechanism, so that the fused representation information is more suitable for clustering representation information.

Description

Technical Field

[0001] The present invention belongs to the technical field of community discovery, and relates to a community discovery method based on attention fusion and enhanced neighborhood information. Background Art

[0002] Community discovery refers to finding community structures with similar characteristics in a network graph to understand their topological structure and attribute information, so as to be applied to tasks such as classification and prediction and serve the real society. Community discovery has important practical significance and has been widely studied and applied in many real network problems.

[0003] In addition to the classic spectral clustering and statistical inference methods, deep learning technology for community discovery has made significant progress in recent years due to its advantages in processing high-dimensional network data. The community partitioning work under deep clustering often uses two technologies, autoencoders and graph autoencoders, to find different intrinsic information of the network. However, in the current field of community partitioning based on deep learning, the main focus is on first-order neighbor nodes and hyperparameter fusion representation information. Therefore, there are problems such as insufficient utilization of neighborhood information and insufficient fusion of representation information, which leads to the final cluster distribution being not robust enough. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a community discovery method based on attention fusion and enhanced neighborhood information, so as to solve the problems of insufficient utilization of neighborhood information and insufficient fusion of representation information existing in deep clustering algorithms.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A community discovery method based on attention fusion and enhanced neighborhood information, the method comprising:

[0007] Prepare the original data of the graph, obtain the enhanced neighborhood structure information representation of the original graph structure, and obtain the attribute information representation of the original graph through the autoencoder;

[0008] The attention mechanism is used to fuse and enhance the representation of neighborhood structure information and attribute information, and the graph clustering result is obtained by calculating the soft assignment;

[0009] A supervised loss function is constructed to optimize the graph clustering result through self-supervised training.

[0010] Furthermore, the original data of the graph includes: given an attribute graph G = (V, E, X), V = {v 1 ,v 2 ,…,v n} represents the node set, E represents the edge set, X=[x 1 ,x 2 ,…,xn ] T Represents the characteristic matrix; the topological structure of the graph G adopts the adjacency matrix A={a ij}, using D = diag (d 1 ,d 2 ,…,d n ) represents the degree matrix of A, where Represents node v i The degree.

[0011] Furthermore, the obtaining of the enhanced neighborhood structure information representation of the original graph structure includes:

[0012] Use a graph attention network instead of a graph convolutional network to get the representation of the i-th node in the l-th graph attention layer:

[0013]

[0014] In the formula, represents the representation of the i-th node in the l-th layer, N i represents the set of neighbor nodes of node i, W l represents the learnable parameters of layer l, represents the representation of the neighbor node j of node i in the l-1th layer, α ij represents the attention weight between node i and its neighbor node j;

[0015] Calculate the attention weight α between node i and its neighbor node j ij :

[0016] α ij =softmax(LeakyReLU(a T (Wx i ||Wx j )))

[0017] Where W and a T represents the learnable parameter matrix, “||” represents the concatenation operation, and LeakyReLU represents the activation function;

[0018] Considering the importance of the t-order neighbor node to the current node, the proximity matrix B is introduced to represent the relationship between the t-order neighbor nodes:

[0019]

[0020] Where M ij is the inverse of the degree of node i, indicating the importance of node i to its neighbor node j. If there is an edge between nodes i and j, then M ij =1 / d i , otherwise M ij =0;

[0021] Based on the proximity matrix B ij Modify the attention weight α ij :

[0022]

[0023] In the formula, B ij Represents the proximity matrix between node i and node j;

[0024] The enhanced neighborhood structure information representation H output by the encoder is used to reconstruct the adjacency matrix through the inner product decoder And the cross entropy loss function is used to minimize the reconstruction loss.

[0025] Furthermore, the method of obtaining the attribute information representation of the original graph through the autoencoder includes: inputting the feature matrix X of the original graph into the autoencoder, passing the output result of the encoder to the decoder, and the output of the decoder is the reconstructed feature matrix, that is, the attribute information representation; in addition, the difference between the input feature matrix and the output feature matrix is ​​minimized by the loss function, so that the autoencoder can extract effective graph attribute information.

[0026] Furthermore, the use of the attention mechanism to fuse the enhanced neighborhood structure information representation and the attribute information representation includes: first, the attribute information representation and the enhanced neighborhood structure information representation are concatenated to obtain a connection feature vector, then the relationship between the connection features is captured by a fully connected layer, and then mapped by a nonlinear activation function LeakyReLu, and finally the concatenated vector is normalized by a softmax function and L2 regularization, thereby obtaining an attention coefficient matrix C; the above process is expressed as:

[0027] C=l 2 (softmax((LeakyReLU((H||Z(W))))

[0028] In the formula, C=[C 1 ||C 2 ] represents the attention coefficient matrix, C 1 , C 2 are the fusion attention coefficient vectors of the enhanced neighborhood structure information representation H and attribute information representation Z, W represents the parameter matrix, l 2 represents L2 regularization;

[0029] The enhanced neighborhood structure information representation H and the attribute information representation Z are fused through the Hadamard inner product to obtain the feature representation F:

[0030] F=C 1 ⊙H+C 2 ⊙Z

[0031] Where, ⊙ represents the Hadamard inner product;

[0032] The graph clustering result is obtained by calculating the soft assignment Q:

[0033]

[0034] In the formula, f i is the i-th row of the feature representation F, which represents the node representation that maps the i-th node to the low-dimensional space; u j The jth cluster center initialized for k-means clustering, q ij represents the probability that node i belongs to cluster j, Q = {q ij}.

[0035] Furthermore, the supervised loss function is expressed as:

[0036]

[0037] In the formula, Represents the loss function of the process of obtaining the attribute information representation of the original graph, represents the loss function of the process of obtaining the enhanced neighborhood structure information representation of the original graph structure, and λ represents a hyperparameter;

[0038] represents the clustering loss function, which is expressed as:

[0039]

[0040] In the formula, is the triple clustering loss function, is the clustering consistency loss function; p ij Indicates q ij The target distribution obtained by square normalization; q ij is the probability that node i belongs to cluster j, Q = {q ij} represents the clustering result of the attention mechanism fusion module for fusing the enhanced neighborhood structure information representation and attribute information representation; is the probability that node i belongs to cluster j, represents the clustering result of the autoencoder module used to obtain the attribute information representation; is the probability that node i belongs to cluster j, Represents the clustering result of the graph neighborhood structure information enhancement module for obtaining enhanced neighborhood structure information representation.

[0041] The beneficial effects of the present invention are as follows: the present invention introduces high-order neighborhood information and graph attention mechanism, which enhances the utilization effect of neighborhood information. The present invention fully considers the importance of different neighbor nodes, obtains more refined topological structure information, and uses the attention mechanism to dynamically fuse the extracted enhanced neighborhood topological structure information and attribute information, so that the fused representation information is more suitable for clustering representation information. In addition, in order to ensure the effect of model training, the present invention designs a self-supervised learning method, which combines self-supervised learning and cluster consistency constraints to ensure the full integration and consistency of information from different views in embedded representation. The present invention effectively improves the clustering effect and representation ability, the clustering structure is clearer and more robust, and the ability to identify different communities is effectively improved, thereby providing more accurate representation and better support for downstream tasks.

[0042] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0044] Figure 1 A flowchart of a community discovery method based on attention fusion and enhanced neighborhood information proposed in one embodiment of the present invention;

[0045] Figure 2 A block diagram of the community discovery model structure. DETAILED DESCRIPTION

[0046] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0047] Community discovery clustering mainly focuses on first-order neighbor nodes and hyperparameter fusion representation information, and there are problems of insufficient utilization of neighborhood information and insufficient fusion of representation information. An embodiment of the present invention provides a community discovery method based on attention fusion and enhanced neighborhood information, which can obtain enhanced neighborhood information and fully fused representation information.

[0048] like Figure 1 As shown, the method specifically comprises the following steps:

[0049] 1. First, a community discovery model based on attention fusion and enhanced neighborhood information is established, and the input and output of the model are clarified. The input includes the original data of the graph, and the output is the clustering result based on the community structure. Among them, the public dataset is selected as the initial data.

[0050] The original data of the graph includes: given an attribute graph G = (V, E, X), where V = {v 1 ,v 2 ,…,v n} is a node set with n nodes, E is an edge set, X = [x 1 ,x 2 ,…,x n ] T is the characteristic matrix. The topological structure of the graph G can be represented by the adjacency matrix A = {a ij}express, If v ij =1, indicating node v i To node v j There is an edge between them. D=diag(d 1 ,d 2 ,…,d n ) represents the degree matrix of A, in Represents node v i The degree.

[0051] like Figure 2 As shown, the community discovery model includes a graph neighborhood structure information enhancement module, a graph attribute information module, an attention mechanism fusion module and a self-training supervision module.

[0052] Among them, the graph neighborhood structure information enhancement module adopts an enhanced topological structure strategy. For adjacency matrices with topological structures, richer topological structure information is obtained by adding high-order neighbor structure information of nodes. Since graph aggregation operations are equivalent to aggregation operations, the graph attention mechanism is added to learn the important weights of different neighbor nodes to the current node. After the above operations, rich enhanced neighborhood structure information can be obtained.

[0053] The graph attribute information module uses an autoencoder (AE) to encode and decode the original graph to extract the attribute information of the graph. Through the structure of the autoencoder, it can effectively capture the potential features of the graph and map the high-dimensional graph data to a low-dimensional space, thereby improving the expression ability of the graph attribute information.

[0054] The attention mechanism fusion module adopts the attention mechanism fusion strategy, which inputs the acquired attribute representation information and enhanced neighborhood structure information representation into the fully connected neural network. By training the neural network, the module can learn the fusion weights between different information sources, and by dynamically adjusting the contribution of each information source, the model can adaptively optimize the information fusion strategy in different tasks. This fusion method helps to improve the accuracy and robustness of community discovery, thereby obtaining a clustering representation that is more suitable for community discovery.

[0055] The self-training supervision module is designed to guide and optimize the learning process of clustering information more effectively. The loss function of the self-training supervision module can promote the system to gradually improve the representation ability between modules during the training process by constraining the clustering results output by the model, ensuring that the features of different modules can work together to improve the overall performance.

[0056] 2. Obtain the enhanced topological structure information representation H of the original graph structure A through the graph neighborhood structure information enhancement module. The graph neighborhood structure information enhancement module enriches the topological structure information of the graph by introducing high-order neighbor relationships of nodes, providing stronger structural support for subsequent community discovery.

[0057] 1) In order to measure the importance of different neighbors to the current node, the graph attention network (GAT) is used instead of the graph convolutional network (GCN). In the graph attention layer of the lth layer, the representation of the i-th node is calculated as follows:

[0058]

[0059] in, represents the representation of the i-th node in the l-th layer, N i W represents the set of neighbor nodes of node i. l represents the learnable parameters of the lth layer.

[0060] 2) The attention weight α between node i and neighbor node j can be calculated through the attention mechanism ij :

[0061] α ij =softmax(LeakyReLU(a T (Wx i ||Wx j )))

[0062] Among them, W and a T is a learnable parameter matrix, “||” represents the concatenation operation, which converts the node feature x i and x j The concatenation is performed in dimension, and LeakyReLU is used as the activation function.

[0063] 3) When calculating the attention weight of the current node, there is often a complex structure in the graph, and the high-order neighbors of the node contain more structural information. In order to consider the importance of the t-order neighbor to the current node, the proximity matrix B is introduced to represent the relationship between the t-order neighbor nodes:

[0064]

[0065] Among them, M ij is the inverse of the degree of node i, indicating the importance of node i to its neighbor node j. If there is an edge between nodes i and j, then M ij =1 / d i , otherwise M ij =0.

[0066] 4) Based on the proximity matrix B ij , attention weight α ij It can be modified as follows:

[0067]

[0068] Among them, B ij Represents the proximity matrix between node i and node j.

[0069] 5) In the graph encoder module, the decoder is used to reconstruct the adjacency matrix A of the topological structure. The encoder's embedding representation H is used to reconstruct the adjacency matrix using the inner product decoder

[0070]

[0071] 6) In order to effectively supervise the training of the graph encoder, the cross entropy loss function is used to minimize the graph reconstruction loss, which enables the graph encoder to better capture the topological structure information. The specific loss function is expressed as:

[0072]

[0073] Among them, A i represents the adjacency matrix of node i, represents the reconstructed adjacency matrix of node i, and N represents the number of nodes.

[0074] 3. Use the autoencoder (AE) to obtain the potential features of the graph’s attribute information and further improve the ability to express the graph’s attribute information. Through the encoding and decoding process of the autoencoder, the potential features of the graph can be extracted and the dimension can be reduced, thereby effectively capturing the important attribute information in the graph data. Specifically, the AE autoencoder is used to encode and decode the original feature matrix X to obtain the graph’s attribute information, including:

[0075] 1) Input the feature matrix X of the original image into the encoder of AE:

[0076]

[0077] Among them, Z (0) =X represents encoder input, represents the output of the encoder layer l, is the parameter matrix learned by the encoder layer l. φ is an activation function similar to RELU.

[0078] 2) Pass the output of the encoder to the decoder:

[0079]

[0080] in, is the learnable parameter matrix of the decoder layer l. The output of the decoder is the reconstructed feature matrix

[0081] 3) Calculate the reconstruction loss of the AE module to measure the difference between the input feature matrix and the reconstructed feature matrix:

[0082]

[0083] The loss function guides model learning by minimizing the difference, ensuring that the encoder can extract effective graph attribute information.

[0084] 4. Obtain sufficient fusion representation information through the attention mechanism fusion module. This module combines the representation of attribute information and the representation of enhanced neighborhood structure information. The weight of each information source is dynamically adjusted through the attention mechanism, so that the model can comprehensively consider the influence of different features, and finally obtain more accurate and robust clustering results, providing more precise graph representation for subsequent downstream tasks.

[0085] 1) First, the embedded representation Z (attribute information representation) and the embedded representation H (enhanced neighborhood structure information representation) are concatenated together to form a connected feature vector. Then, the relationship between the connected features is captured through the fully connected layer, and then mapped through the nonlinear activation function LeakyReLu. Finally, the concatenated vector is normalized using softmax and L2 regularization to obtain the attention coefficient matrix C. The process can be expressed as:

[0086] C=l 2 (softmax((LeakyReLU((H||Z)W))))

[0087] Where C = [C 1 ||C 2 ] is the attention coefficient matrix, C 1 , C 2are the fused attention coefficient vectors of H and Z, respectively, and W is the learned parameter matrix. 2 represents the regularization operation.

[0088] 2) H and Z are fused through the Hadamard product to obtain the final feature representation F:

[0089] F=C 1 ⊙H+C 2 ⊙Z

[0090] Where ⊙ represents the Hadamard inner product operation.

[0091] The final fused feature representation F contains important information from the GATE module and the AE module. By introducing this attention mechanism, the fused representation can more effectively capture the key information from the two modules, thereby improving the performance of clustering and community discovery, and providing more favorable feature support for subsequent tasks.

[0092] 5. The community discovery task of graph clustering is an unsupervised task, and there is a lack of strong supervisory information to guide the optimization of clustering during the training process. Therefore, in order to improve the clustering effect and the representation ability of the module, this embodiment effectively guides and optimizes the learning process of clustering information through a self-supervised training module. This module can gradually improve the representation ability between modules during the training process by designing a supervised loss function, ensuring that the features of different modules can work together, thereby optimizing the clustering results and improving the performance of the overall model.

[0093] 1) Use Student's t distribution to calculate the similarity between the fusion representation F, the attribute information representation Z, and the enhanced neighborhood structure information representation H cluster centers, and calculate the soft assignment Q, Q z and Q h The clustering results of the attention mechanism fusion module, the autoencoder module, and the graph neighborhood structure information enhancement module are obtained. The specific calculation formula is as follows:

[0094]

[0095] Among them, f i is the i-th row of the fused representation F, z i is the i-th row of the embedding representation Z, h i is the i-th row of the embedding representation H, which respectively represents the node representation that maps the i-th node to the low-dimensional space. j is the jth cluster center initialized by k-means. ij represents the probability that node i belongs to cluster j, Q = {q ij}, Same reason.

[0096] 2) In order to improve the confidence of training and avoid collapse, the calculated soft assignment matrix Q is squared and normalized to generate the target distribution P for supervision. The specific calculation formula is as follows:

[0097]

[0098] 3) In order to make the model learn a more suitable cluster distribution and simplify the model training process, the triple clustering loss function is introduced for optimization. The loss function is expressed as:

[0099]

[0100] 4) Based on the consistency assumption of data object clusters, it is assumed that the cluster structure of the network topology extracted by the enhanced domain module and the node cluster structure extracted by the AE module have potential consistency. In order to maintain this consistency, this embodiment introduces a clustering consistency loss function and uses KL divergence to reduce the distribution difference between the two modules. The specific formula is as follows:

[0101]

[0102] 5) Integrate the triplet clustering loss and cluster consistency loss together to jointly promote model optimization and generate a more robust cluster distribution. The final clustering loss function is as follows:

[0103]

[0104] 6) In order to balance the reconstruction loss and clustering loss, the hyperparameter λ is introduced and applied to the overall optimization objective of the model. The final optimization objective loss function is:

[0105]

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A community discovery method based on attention fusion and enhanced neighborhood information, characterized in that: The method comprises: preparing original data of a graph, obtaining an enhanced neighborhood structure information representation of the original graph structure, and obtaining an attribute information representation of the original graph through an autoencoder; The attention mechanism is used to fuse and enhance the representation of neighborhood structure information and attribute information, and the graph clustering result is obtained by calculating the soft assignment; A supervised loss function is constructed to optimize the graph clustering result through self-supervised training.

2. The community discovery method according to claim 1, characterized in that: The original data of the graph includes: given an attribute graph G = (V, E, X), V = {v1, v2, ..., v n } represents the node set, E represents the edge set, X=[x1,x2,…,x n ] T represents the feature matrix; The topological structure of the graph G adopts the adjacency matrix A = {a ij }, using D = diag (d1, d2, ..., d n ) represents the degree matrix of A, where Represents node v i The degree.

3. The community discovery method according to claim 1, characterized in that: The obtaining of the enhanced neighborhood structure information representation of the original graph structure includes: Use a graph attention network instead of a graph convolutional network to get the representation of the i-th node in the l-th graph attention layer: In the formula, represents the representation of the i-th node in the l-th layer, N i represents the set of neighbor nodes of node i, W l represents the learnable parameters of layer l, represents the representation of the neighbor node j of node i in the l-1th layer, α ij represents the attention weight between node i and its neighbor node j; Calculate the attention weight α between node i and its neighbor node j ij : α ij =softmax(LeakyReLU(a T (Wx i ||Wx j ))) Where W and a T represents the learnable parameter matrix, "||" represents the concatenation operation, and LeakyReLU represents the activation function; Considering the importance of the t-order neighbor node to the current node, the proximity matrix B is introduced to represent the relationship between the t-order neighbor nodes: Where M ij is the inverse of the degree of node i, indicating the importance of node i to its neighbor node j. If there is an edge between nodes i and j, then M ij =1 / d i , otherwise M ij =0,d i represents the degree of the i-th node; Based on the proximity matrix B ij Modify the attention weight α ij : In the formula, B ij Represents the proximity matrix between node i and node j; The enhanced neighborhood structure information representation H output by the encoder is used to reconstruct the adjacency matrix through the inner product decoder And the cross entropy loss function is used to minimize the reconstruction loss.

4. The community discovery method according to claim 1, characterized in that: The method of obtaining the attribute information representation of the original graph through the autoencoder includes: inputting the feature matrix X of the original graph into the autoencoder, passing the output result of the encoder to the decoder, and the output of the decoder is the reconstructed feature matrix, that is, the attribute information representation; in addition, the difference between the input feature matrix and the output feature matrix is ​​minimized by the loss function, so that the autoencoder can extract effective graph attribute information.

5. The community discovery method according to claim 1, characterized in that: The method of using the attention mechanism to fuse the enhanced neighborhood structure information representation and the attribute information representation includes: first, concatenating the attribute information representation and the enhanced neighborhood structure information representation to obtain a connection feature vector, then capturing the relationship between the connection features through a fully connected layer, and then mapping through a nonlinear activation function LeakyReLu, and finally normalizing the concatenated vector using a softmax function and L2 regularization to obtain an attention coefficient matrix C; the above process is expressed as: C=l2(softmax((LeakyReLU((H||Z(W)))) In the formula, C = [C1||C2] represents the attention coefficient matrix, C1 and C2 are the fused attention coefficient vectors of the enhanced neighborhood structure information representation H and attribute information representation Z, W represents the parameter matrix, and l2 represents L2 regularization; The enhanced neighborhood structure information representation H and the attribute information representation Z are fused through the Hadamard inner product to obtain the feature representation F: F=C1⊙H+C2⊙Z Where, ⊙ represents the Hadamard inner product; The graph clustering result is obtained by calculating the soft assignment Q: In the formula, f i is the i-th row of the feature representation F, which represents the node representation that maps the i-th node to the low-dimensional space; u j The jth cluster center initialized for k-means clustering, q ij represents the probability that node i belongs to cluster j, Q = {q ij }.

6. The community discovery method according to claim 1, characterized in that: The supervised loss function is expressed as: In the formula, Represents the loss function of the process of obtaining the attribute information representation of the original graph, represents the loss function of the process of obtaining the enhanced neighborhood structure information representation of the original graph structure, and λ represents a hyperparameter; represents the clustering loss function, which is expressed as: In the formula, is the triple clustering loss function, is the clustering consistency loss function; p ij Indicates q ij The target distribution obtained by square normalization; q ij is the probability that node i belongs to cluster j, Q = {q ij } represents the clustering result of the attention mechanism fusion module for fusing the enhanced neighborhood structure information representation and attribute information representation; is the probability that node i belongs to cluster j, represents the clustering result of the autoencoder module used to obtain the attribute information representation; is the probability that node i belongs to cluster j, Represents the clustering result of the graph neighborhood structure information enhancement module for obtaining enhanced neighborhood structure information representation.

Images