A Community Discovery Method Based on Attention Fusion and Enhanced Neighborhood Information

Abstract

This invention relates to a community detection method based on attention fusion and enhanced neighborhood information, belonging to the field of community detection technology. The method includes: preparing raw graph data; obtaining enhanced neighborhood structure information representations of the original graph structure; and simultaneously obtaining attribute information representations of the original graph through an autoencoder; fusing the enhanced neighborhood structure information representations and attribute information representations using an attention mechanism; obtaining graph clustering results through soft assignment calculation; constructing a supervised loss function; and optimizing the graph clustering results through self-supervised training. This invention introduces high-order neighborhood information and a graph attention mechanism, enhancing the utilization of neighborhood information. Simultaneously, this invention fully considers the importance of different neighbor nodes, obtaining more refined topological structure information, and dynamically fusing the extracted enhanced neighborhood topological structure information and attribute information using an attention mechanism, making the fused representation information more suitable for clustering representation information.

Description

Technical Field

[0001] This invention belongs to the field of community discovery technology and relates to a community discovery method based on attention fusion and enhanced neighborhood information. Background Technology

[0002] Community detection refers to the process of identifying community structures with similar characteristics in network graphs to understand their topological structure and attribute information. This information can then be applied to tasks such as classification and prediction, serving real-world applications. Community detection has significant practical implications and has been widely researched and applied to many real-world network problems.

[0003] Besides classic spectral clustering and statistical inference methods, deep learning techniques for community detection have seen significant development in recent years due to their advantages in handling high-dimensional network data. Community partitioning in deep clustering often uses autoencoders and graph autoencoders to find the intrinsic information that differentiates the network. However, current deep learning-based community partitioning primarily focuses on first-order neighbor nodes and hyperparameter fusion representation information, resulting in insufficient utilization of neighborhood information and inadequate fusion of representation information. This leads to a less robust final cluster distribution. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a community discovery method based on attention fusion and enhanced neighborhood information, so as to solve the problems of insufficient utilization of neighborhood information and insufficient fusion of representation information in deep clustering algorithms.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A community detection method based on attention fusion and enhanced neighborhood information, the method comprising:

[0007] Prepare the raw data of the graph, obtain the enhanced neighborhood structure information representation of the original graph structure, and obtain the attribute information representation of the original graph through an autoencoder;

[0008] An attention mechanism is used to fuse enhanced neighborhood structure information representation and attribute information representation, and graph clustering results are obtained by computational soft assignment;

[0009] A supervised loss function is constructed, and the graph clustering results are optimized through self-supervised training.

[0010] Furthermore, the original data of the graph includes, given an attribute graph G = (V, E, X), V = {v1, v2, ..., v...} n Let} represent the node set, E represent the edge set, and X = [x1, x2, ..., x...]. n ] T The characteristic matrix is ​​represented; the topological structure of the graph G is represented by the adjacency matrix A = {aij} indicates that D = diag(d1, d2, ..., d n ) represents the degree matrix of A, where Represents node v i The degree.

[0011] Furthermore, the enhanced neighborhood structure information representation of the original graph structure includes:

[0012] Instead of a graph convolutional network, a graph attention network is used to obtain the representation of the i-th node in the l-th graph attention layer:

[0013]

[0014] In the formula, This represents the representation of the i-th node in the l-th layer, N. i Let W represent the set of neighboring nodes of node i. l This represents the learnable parameters of the l-th layer. Let α represent the representation of node j, a neighbor of node i in the (l-1)th layer. ij This represents the attention weight between node i and its neighbor node j;

[0015] Calculate the attention weight α between node i and its neighbor node j. ij :

[0016] α ij =softmax(LeakyReLU(a T (Wx i ||Wx j )))

[0017] In the formula, W and a T This represents the learnable parameter matrix, "||" represents the concatenation operation, and LeakyReLU represents the activation function.

[0018] Considering the importance of t-order neighbor nodes to the current node, a proximity matrix B is introduced to represent the relationships between t-order neighbor nodes:

[0019]

[0020] In the formula, M ij M is the reciprocal of the degree of node i, representing the importance of node i to its neighbor node j. If there is an edge between nodes i and j, then M... ij =1 / d i Otherwise M ij =0;

[0021] Based on the proximity matrix B ij Modify attention weight α ij :

[0022]

[0023] In the formula, B ij This represents the proximity matrix between node i and node j;

[0024] The enhanced neighborhood structure information H output by the encoder is used to reconstruct the adjacency matrix through the inner product decoder. The reconstruction loss is minimized using the cross-entropy loss function.

[0025] Furthermore, the step of obtaining the attribute information representation of the original graph through the autoencoder includes: inputting the feature matrix X of the original graph into the autoencoder, passing the output of the encoder to the decoder, and the output of the decoder being the reconstructed feature matrix, which is the attribute information representation; in addition, minimizing the difference between the input feature matrix and the output feature matrix through a loss function enables the autoencoder to extract effective graph attribute information.

[0026] Furthermore, the method of fusing enhanced neighborhood structure information representation and attribute information representation using an attention mechanism includes: firstly, concatenating the attribute information representation and the enhanced neighborhood structure information representation to obtain a connection feature vector; then, capturing the relationship between the connection features through a fully connected layer; next, mapping using the nonlinear activation function LeakyReLU; and finally, normalizing the concatenated vector using the softmax function and L2 regularization to obtain the attention coefficient matrix C; the above process is expressed as:

[0027] C=l2(softmax((LeakyReLU((H||Z(W))))

[0028] In the formula, C = [C1||C2] represents the attention coefficient matrix, C1 and C2 are the fusion attention coefficient vectors of the enhanced neighborhood structural information representation H and the attribute information representation Z, respectively, W represents the parameter matrix, and l2 represents L2 regularization;

[0029] The feature representation F is obtained by fusing the enhanced neighborhood structure information representation H and the attribute information representation Z through the Hadamard inner product:

[0030] F = C1⊙H + C2⊙Z

[0031] In the formula, ⊙ represents the Hadamah inner product;

[0032] The graph clustering results are obtained by calculating the soft assignment Q:

[0033]

[0034] In the formula, f i Let be the i-th row of the feature representation F, representing the node representation that maps the i-th node to the low-dimensional space; uj The j-th cluster center initialized for k-means clustering, q ij Let Q represent the probability that node i belongs to cluster j, where Q = {q} ij}

[0035] Furthermore, the supervised loss function is expressed as:

[0036]

[0037] In the formula, This represents the loss function used to obtain the attribute information of the original graph. The loss function represents the process of obtaining enhanced neighborhood structure information representation of the original graph structure, where λ represents the hyperparameter.

[0038] The clustering loss function is expressed as:

[0039]

[0040] In the formula, Let triplet clustering loss function be used. p is the clustering consistency loss function; ij Indicates q ij The target distribution obtained by square normalization; q ij Let Q = {q} be the probability that node i belongs to cluster j. ij} represents the clustering result of the attention mechanism fusion module used to fuse the enhanced neighborhood structure information representation and attribute information representation; Let be the probability that node i belongs to cluster j. This represents the clustering results of the autoencoder module used to obtain attribute information representations; Let be the probability that node i belongs to cluster j. This represents the clustering results of the graph neighborhood structure information enhancement module used to obtain enhanced neighborhood structure information representations.

[0041] The beneficial effects of this invention are as follows: This invention introduces higher-order neighborhood information and a graph attention mechanism, enhancing the utilization of neighborhood information. This invention fully considers the importance of different neighboring nodes, obtaining more refined topological structure information. It dynamically fuses the extracted enhanced neighborhood topological structure information and attribute information using an attention mechanism, making the fused representation information more suitable for clustering representation. Furthermore, to ensure the effectiveness of model training, this invention designs a self-supervised learning method, combining self-supervised learning and clustering consistency constraints to guarantee the full fusion and consistency of information from different views in the embedded representation. This invention effectively improves clustering results and representation capabilities, resulting in a clearer and more robust clustering structure, effectively enhancing the ability to identify different communities, thereby providing more accurate representations and better support for downstream tasks.

[0042] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0043] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0044] Figure 1 This is a schematic diagram of the community discovery method based on attention fusion and enhanced neighborhood information proposed in an embodiment of the present invention;

[0045] Figure 2 Discover the model structure diagram for the community. Detailed Implementation

[0046] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0047] Community discovery clustering mainly focuses on first-order neighbor nodes and hyperparameter fusion representation information, which suffers from insufficient utilization of neighborhood information and insufficient fusion of representation information. An embodiment of the present invention provides a community discovery method based on attention fusion and enhanced neighborhood information, which can obtain enhanced neighborhood information and fully fused representation information.

[0048] like Figure 1 As shown, the method specifically includes the following steps:

[0049] 1. First, a community discovery model based on attention fusion and enhanced neighborhood information is established, and the model's input and output are defined. The input includes the original graph data, and the output is the clustering results based on community structure. A public dataset is selected as the initial data.

[0050] The original data for the graph includes a given attribute graph G = (V, E, X), where V = {v1, v2, ..., v...} n} is a node set with n nodes, E is an edge set, and X = [x1, x2, ..., xn]. n ] T The characteristic matrix is ​​used. The topological structure of graph G can be represented by the adjacency matrix A = {a...} ij}express, If v ij =1, indicating node v i to node v j There exists an edge between them. D = diag(d1, d2, ..., d... n ) represents the degree matrix of A. in Represents node v i The degree.

[0051] like Figure 2 As shown, the community detection model includes a graph neighborhood structure information enhancement module, a graph attribute information module, an attention mechanism fusion module, and a self-training supervision module.

[0052] The graph neighborhood structure information enhancement module employs a strategy of enhancing topological structure. For adjacency matrices with topological structure, richer topological structure information is obtained by incorporating higher-order neighbor structure information of nodes. Since graph aggregation operations are equivalent to aggregation operations, a graph attention mechanism is added to learn the important weights of different neighbor nodes on the current node. Through these operations, rich and enhanced neighborhood structure information can be obtained.

[0053] The graph attribute information module uses an autoencoder (AE) to encode and decode the original graph, thereby extracting its attribute information. The structure of the autoencoder effectively captures the latent features of the graph and maps high-dimensional graph data to a low-dimensional space, thus improving the expressive power of graph attribute information.

[0054] The attention mechanism fusion module employs an attention mechanism fusion strategy, inputting the acquired attribute representations and enhanced neighborhood structure information into a fully connected neural network. By training the neural network, this module learns the fusion weights between different information sources. By dynamically adjusting the contribution of each information source, the model can adaptively optimize the information fusion strategy across different tasks. This fusion approach helps improve the accuracy and robustness of community detection, thereby obtaining a clustering representation more suitable for community detection.

[0055] The self-training supervised module aims to more effectively guide and optimize the learning process of clustering information. By constraining the clustering results output by the model, the loss function of the self-training supervised module promotes the gradual improvement of the representation capabilities between modules during training, ensuring that the features of different modules can work synergistically, thereby improving overall performance.

[0056] 2. The graph neighborhood structure information enhancement module is used to obtain an enhanced topological structure representation H of the original graph structure A. This module enriches the graph's topological structure information by introducing higher-order neighbor relationships, providing stronger structural support for subsequent community discovery.

[0057] 1) To measure the importance of different neighbors to the current node, a Graph Attention Network (GAT) is used instead of a Graph Convolutional Network (GCN). In the l-th layer of the Graph Attention Network, the representation of the i-th node is calculated as follows:

[0058]

[0059] in, This represents the representation of the i-th node in the l-th layer, N. i Let W represent the set of neighboring nodes of node i. l This represents the learnable parameters of the l-th layer.

[0060] 2) The attention weight α between node i and its neighbor node j can be calculated using the attention mechanism. ij :

[0061] α ij =softmax(LeakyReLU(a T (Wx i ||Wx j )))

[0062] Among them, W and a T It is a learnable parameter matrix, where "||" represents the concatenation operation, combining the node features x. i and x j The concatenation is performed along the dimension, and LeakyReLU is the activation function.

[0063] 3) When calculating the attention weight of the current node, the graph often contains complex structures, and the node's higher-order neighbors contain more structural information. To consider the importance of t-order neighbors to the current node, a proximity matrix B is introduced to represent the relationships between t-order neighbor nodes:

[0064]

[0065] Among them, M ij M is the reciprocal of the degree of node i, representing the importance of node i to its neighbor node j. If there is an edge between nodes i and j, then M... ij =1 / d i Otherwise M ij =0.

[0066] 4) Based on the proximity matrix B ij Attention weight α ijIt can be modified to the following form:

[0067]

[0068] Among them, B ij This represents the proximity matrix between node i and node j.

[0069] 5) In the graph encoder module, the decoder's role is to reconstruct the adjacency matrix A of the topological structure, using the inner product decoder to rebuild the adjacency matrix from the encoder's embedding representation H.

[0070]

[0071] 6) To effectively supervise the training of the graph encoder, a cross-entropy loss function is used to minimize the graph reconstruction loss, which allows the graph encoder to better capture topological information. The specific loss function is expressed as follows:

[0072]

[0073] Among them, A i This represents the adjacency matrix of node i. Let N represent the reconstructed adjacency matrix of node i, and let N represent the number of nodes.

[0074] 3. Use an autoencoder (AE) to obtain latent features of graph attributes, further enhancing the expressive power of graph attribute information. Through the encoding and decoding process of the autoencoder, latent features of the graph can be extracted and dimensionality reduced, thereby effectively capturing important attribute information in the graph data. Specifically, the AE autoencoder is used to encode and decode the original feature matrix X to obtain the graph's attribute information, including:

[0075] 1) Input the feature matrix X of the original image into the encoder of the AE:

[0076]

[0077] Among them, Z (0) =X represents the encoder input, This represents the output of the encoder's l-th layer. Let be the parameter matrix learned by the encoder at layer l. φ is an activation function similar to ReLU.

[0078] 2) Pass the encoder's output to the decoder:

[0079]

[0080] in, Let be the learnable parameter matrix of the l-th layer of the decoder. The output of the decoder is the reconstructed feature matrix.

[0081] 3) Calculate the reconstruction loss of the AE module to measure the difference between the input feature matrix and the reconstructed feature matrix:

[0082]

[0083] The loss function guides model learning by minimizing this difference, ensuring that the encoder can extract effective graph attribute information.

[0084] 4. Sufficient fusion representation information is obtained through the attention mechanism fusion module. This module combines attribute information representation and enhanced neighborhood structure information representation, dynamically adjusting the weights of each information source through the attention mechanism. This allows the model to comprehensively consider the influence of different features, ultimately obtaining more accurate and robust clustering results, providing a more precise graph representation for subsequent downstream tasks.

[0085] 1) First, the embedding representation Z (attribute information representation) and the embedding representation H (enhanced neighborhood structure information representation) are concatenated to form a connection feature vector. Then, the relationships between the connection features are captured through a fully connected layer, followed by mapping using the non-linear activation function LeakyReLu. Finally, softmax and L2 regularization are used to normalize the concatenated vector, resulting in the attention coefficient matrix C. This process can be represented as:

[0086] C=l2(softmax((LeakyReLU((H||Z)W))))

[0087] Where C = [C1||C2] is the attention coefficient matrix, C1 and C2 are the fused attention coefficient vectors of H and Z respectively, and W is the learned parameter matrix. l2 represents the regularization operation.

[0088] 2) The final feature representation F is obtained by fusing H and Z through the Hadamard inner product:

[0089] F = C1⊙H + C2⊙Z

[0090] Here, ⊙ represents the Hadamard inner product operation.

[0091] The final fused feature representation F incorporates key information from both the GATE and AE modules. By introducing this attention mechanism, the resulting fused representation can more effectively capture crucial information from both modules, thereby improving the performance of clustering and community detection and providing more favorable feature support for subsequent tasks.

[0092] 5. Graph clustering community detection is an unsupervised task, lacking strong supervisory information to guide cluster optimization during training. Therefore, to improve clustering performance and module representation capabilities, this embodiment employs a self-supervised training module to effectively guide and optimize the learning process of clustering information. This module, through the design of a supervised loss function, gradually improves the representation capabilities of each module during training, ensuring that features from different modules can synergistically enhance the clustering results and improve the overall model performance.

[0093] 1) Calculate the similarity between cluster centers using the Student t-distribution, the fusion representation F, the attribute information representation Z, and the enhanced neighborhood structure information representation H, and then calculate the soft assignments Q and Q'. z and Q h The clustering results of the attention mechanism fusion module, autoencoder module, and graph neighborhood structure information enhancement module are obtained. The specific calculation formula is as follows:

[0094]

[0095] Among them, f i It is the i-th row of the fusion representation F, z i For the i-th row of the embedded representation Z, h i For the i-th row of the embedding representation H, they represent the node representations that map the i-th node to the low-dimensional space. j q is the j-th cluster center initialized by k-means. ij Let Q represent the probability that node i belongs to cluster j, where Q = {q} ij}, Similarly.

[0096] 2) To improve training confidence and avoid crash phenomena, the calculated soft allocation matrix Q is squared-normalized to generate the target distribution P for supervision. The specific calculation formula is as follows:

[0097]

[0098] 3) To enable the model to learn a more suitable clustering distribution and simplify the model training process, a triplet clustering loss function is introduced for optimization. This loss function is expressed as:

[0099]

[0100] 4) Based on the consistency assumption of data object clusters, it is assumed that the cluster structure of the network topology extracted by the enhancement domain module and the node cluster structure extracted by the AE module have potential consistency. To maintain this consistency, this embodiment introduces a clustering consistency loss function and uses KL divergence to reduce the distribution differences between the two modules. The specific formula is as follows:

[0101]

[0102] 5) The triplet clustering loss and cluster consistency loss are integrated to jointly drive model optimization and generate a more robust cluster distribution. The final clustering loss function is as follows:

[0103]

[0104] 6) To balance reconstruction loss and clustering loss, a hyperparameter λ is introduced and applied to the overall optimization objective of the model. The final objective loss function is:

[0105]

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A community discovery method based on attention fusion and enhanced neighborhood information, characterized in that, The method includes: preparing the original data of the graph, obtaining the enhanced neighborhood structure information representation of the original graph structure, and obtaining the attribute information representation of the original graph through an autoencoder; An attention mechanism is used to fuse enhanced neighborhood structure information representation and attribute information representation, and graph clustering results are obtained by computational soft assignment; A supervised loss function is constructed, and the graph clustering results are optimized through self-supervised training; The attention mechanism that fuses enhanced neighborhood structure information representation and attribute information representation includes: concatenating the attribute information representation and the enhanced neighborhood structure information representation to obtain a connection feature vector; then capturing the relationships between the connection features through a fully connected layer; mapping using the non-linear activation function LeakyReLU; and finally normalizing the concatenated vector using the softmax function and L2 regularization to obtain the attention coefficient matrix C; the above process is expressed as: C=l2(softmax((LeakyReLU((H||Z)W)))) In the formula, C = [C1||C2] represents the attention coefficient matrix, C1 and C2 are the fusion attention coefficient vectors of the enhanced neighborhood structural information representation H and the attribute information representation Z, respectively, W represents the parameter matrix, and l2 represents L2 regularization; The feature representation F is obtained by fusing the enhanced neighborhood structure information representation H and the attribute information representation Z through the Hadamard inner product: F = C1⊙H + C2⊙Z In the formula, ⊙ represents the Hadamah inner product; The graph clustering results are obtained by calculating the soft assignment Q: In the formula, f i Let be the i-th row of the feature representation F, representing the node representation that maps the i-th node to the low-dimensional space; u j The j-th cluster center initialized for k-means clustering, q ij Let Q represent the probability that node i belongs to cluster j, where Q = {q} ij }; The supervision loss function is expressed as: In the formula, This represents the loss function used to obtain the attribute information of the original graph. The loss function represents the process of obtaining enhanced neighborhood structure information representation of the original graph structure, where λ represents the hyperparameter. The clustering loss function is expressed as: In the formula, The loss function for triple clustering is... p is the clustering consistency loss function; ij Indicates q ij The target distribution obtained by square normalization; q ij Let Q = {q} be the probability that node i belongs to cluster j. ij } represents the clustering result of the attention mechanism fusion module used to fuse the enhanced neighborhood structure information representation and attribute information representation; Let be the probability that node i belongs to cluster j. This represents the clustering results of the autoencoder module used to obtain attribute information representations; Let be the probability that node i belongs to cluster j. This represents the clustering results of the graph neighborhood structure information enhancement module used to obtain enhanced neighborhood structure information representations.

2. The community detection method according to claim 1, characterized in that, The original data of the graph includes, given an attribute graph G = (V, E, X), V = {v1, v2, ..., v n Let} represent the node set, E represent the edge set, and X = [x1, x2, ..., x...]. n ] T Represents the characteristic matrix; The topological structure of graph G uses an adjacency matrix A = {a ij } indicates that D = diag(d1, d2, ..., d n ) represents the degree matrix of A, where Represents node v i The degree.

3. The community detection method according to claim 1, characterized in that, The enhanced neighborhood structure information representation of the original graph structure includes: Instead of a graph convolutional network, a graph attention network is used to obtain the representation of the i-th node in the l-th graph attention layer: In the formula, This represents the representation of the i-th node in the l-th layer, N. i Let W represent the set of neighboring nodes of node i. l This represents the learnable parameters of the l-th layer. Let α represent the representation of node j, a neighbor of node i in the (l-1)th layer. ij This represents the attention weight between node i and its neighbor node j; Calculate the attention weight α between node i and its neighbor node j. ij : α ij =softmax(LeakyReLU(a T (Wx i ||Wx j ))) In the formula, W and a T represents the learnable parameter matrix, "||" represents the concatenation operation, and LeakyReLU represents the activation function; Considering the importance of t-order neighbor nodes to the current node, a proximity matrix B is introduced to represent the relationships between t-order neighbor nodes: In the formula, M ij M is the reciprocal of the degree of node i, representing the importance of node i to its neighbor node j. If there is an edge between nodes i and j, then M... ij =1 / d i Otherwise M ij =0,d i This represents the degree of the i-th node; Based on the proximity matrix B ij Modify attention weight α ij : In the formula, B ij This represents the proximity matrix between node i and node j; The enhanced neighborhood structure information H output by the encoder is used to reconstruct the adjacency matrix through the inner product decoder. The reconstruction loss is minimized using the cross-entropy loss function.

4. The community detection method according to claim 1, characterized in that, The method of obtaining attribute information representation of the original graph through autoencoder includes: inputting the feature matrix X of the original graph into the autoencoder, passing the output of the encoder to the decoder, and the output of the decoder being the reconstructed feature matrix, which is the attribute information representation; in addition, minimizing the difference between the input feature matrix and the output feature matrix through a loss function, so that the autoencoder can extract effective graph attribute information.

Images