Community Detection Method, System, Device, and Medium Based on Graph Dual Autoencoder

Abstract

This invention discloses a community detection method, system, device, and medium based on a graph dual autoencoder, relating to the field of community detection technology. The method includes inputting a given citation network into a graph dual autoencoder to obtain graph structure representation information and graph attribute representation information; fusing the graph structure representation information and graph attribute representation information to obtain fused graph representation information; and using a clustering method to divide the fused graph representation information into communities to obtain the community detection result. This invention improves the accuracy of community division in citation networks.

Description

Technical Field

[0001] This invention relates to the field of community discovery technology, and in particular to a community discovery method, system, device and medium based on a graph dual autoencoder. Background Technology

[0002] Citation analysis refers to the analysis of citations and being cited by scientific journals, papers, and authors to reveal their quantitative characteristics and inherent patterns. Citation analysis is highly useful for research topic selection, research hotspots, research trends within a field, finding high-impact scientists, and literature review. The most common citation analysis tools are Web of Science, Scopus, and Google Scholar; however, these three tools are generally only useful for calculating journal impact factors, the number of citations for a particular article, and the citations for a particular author. For ordinary researchers, especially for understanding the content of their research, they are not so practical. An excellent citation analysis method can discover important literature from the perspective of literature citation and explore the flow of scientific knowledge. Graphs are a general data structure for exploring and modeling complex systems in the real world. As an important medium for the interaction of entity relationships, they have also become one of the current research hotspots. Complex networks are typically represented by a graph consisting of a set of nodes (vertices) connected by edges. Citation networks are a type of complex network where each node represents a citation. An edge between two nodes indicates a citation relationship between the two citations; the absence of an edge indicates no citation relationship. Community detection refers to finding communities with similar characteristics within a network graph to understand their topology and attributes, which can then be applied to tasks such as classification and prediction, serving real-world applications. Community detection has significant practical implications and has been widely studied and applied in many real-world network problems. Exploring the community structure of citation networks is beneficial to the citation analysis process, significantly contributing to the discovery of important documents and the exploration of scientific knowledge flows. Therefore, a good community detection method plays a crucial role in the development of the field of citation analysis.

[0003] With the emergence of complex networks, these networks not only possess a large number of nodes but also diverse node features and important attribute information. This poses a challenge to traditional community detection methods. Traditional methods primarily process the structural information of the graph without fully exploring the content within the attribute information. These methods have achieved good results on networks without node features, but when faced with large-scale network datasets (such as citation networks), how to simultaneously preserve network structural information and node attribute information to detect community structures in complex networks has become an emerging research task. Graph neural networks are an application and innovation of traditional deep learning methods on graph structure data, used to extract feature representations from graphs. This technology has made up for the shortcomings of traditional methods. Autoencoders (AEs), as an unsupervised learning artificial neural network, are widely used in feature extraction. Their success in image processing has led researchers to explore the application of autoencoders for community detection.

[0004] Most recent community detection methods based on graph autoencoders (GAs) employ structural reconstruction, while a few that use feature reconstruction still rely on conventional architectures. This lossless feature reconstruction can be unreliable, resulting in less robust models. Therefore, existing GAs have significant room for improvement in solving the community detection problem. In recent years, several GA dual autoencoder models have emerged in the community detection field, reconstructing both structural and attribute features. These algorithms exhibit good graph representation learning capabilities and community partitioning performance, demonstrating the potential of using GA dual autoencoders for community partitioning.

[0005] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0006] (1) The network topology and node attribute features cannot be considered at the same time. Traditional community detection methods mainly include statistical inference methods and machine learning methods. These methods are based on the structural features of the network and only consider the edge relationships between nodes to divide the community, while ignoring the features of the nodes themselves, resulting in a lack of semantics in community division. There are other classic methods, such as K-Means, which only use node attributes to detect communities. These methods ignore the relationships between nodes, i.e., the structural features of the network.

[0007] (2) Most recent community detection methods based on graph autoencoders adopt a single approach of reconstructing the structure or features, which does not fully learn the graph representation. Most of these methods choose structural reconstruction and place too much emphasis on structural information; a small number of encoders that use feature reconstruction still use ordinary architectures. This undamaged feature reconstruction may be unreliable, and the designed models have the problem of weak robustness.

[0008] (3) Most targets in the graph are feature vectors with less information. Multilayer perceptrons, which are often used as decoders in graph autoencoders, may not be able to bridge the gap between encoder representation and decoder target, and cannot obtain graph features well. Such poor graph representation information is not conducive to subsequent community division.

[0009] (4) The mean squared error (MSE) used by the graph autoencoder with feature reconstruction when calculating the reconstruction loss is affected by the existence of different feature vector norms and dimensions, which may lead to model instability.

[0010] The aforementioned problems constrain the development of community discovery methods, and further restrict the progress of citation analysis techniques. Summary of the Invention

[0011] The purpose of this invention is to provide a community discovery method, system, device, and medium based on a graph dual autoencoder, which improves the accuracy of community segmentation in citation networks.

[0012] To achieve the above objectives, the present invention provides the following solution:

[0013] A community detection method based on a graph dual autoencoder includes:

[0014] The given citation network is input into a graph dual autoencoder to obtain graph structure representation information and graph attribute representation information;

[0015] The graph structure representation information and the graph attribute representation information are fused to obtain the fused graph representation information;

[0016] Clustering methods are used to divide the fused graph representation information into communities, resulting in community discovery results.

[0017] Optionally, the graphical dual autoencoder includes a first encoder and a second encoder;

[0018] The first encoder is used to output the features of each node after fusing neighborhood information based on the attention coefficients of each node and its neighboring nodes, as well as the node features of the neighboring nodes, to obtain graph structure representation information; the nodes are nodes in the citation network.

[0019] The second encoder is used to sample nodes in the citation network using a random sampling strategy to obtain a sample set. The features of the nodes in the sample set are masked using a first mask token. Graph information learning is performed on the nodes after the first mask token masking process and the nodes after the first mask token masking process, to obtain graph attribute representation information.

[0020] Optionally, the graph autoencoder for the reconstructed structure employs a graph attention network; the graph autoencoder for the reconstructed features employs a graph neural network.

[0021] Optionally, the community detection method based on graph dual autoencoder further includes training the graph dual autoencoder; the loss function used to train the graph dual autoencoder includes structural reconstruction loss, feature reconstruction loss and clustering loss.

[0022] Optionally, the graph dual autoencoder further includes a first decoder and a second decoder, wherein the first decoder is a decoder of the graph autoencoder for reconstructing the structure, and the second decoder is a decoder of the graph autoencoder for reconstructing the features.

[0023] The first decoder is used to perform an inner product operation on the graph structure representation information to obtain the reconstructed adjacency matrix;

[0024] The second decoder is used for:

[0025] The nodes that have been masked by the first mask token are remasked using the second mask token;

[0026] For nodes after remasking, based on the neighboring nodes of the remasked nodes, a graph neural network is used to reconstruct the features of the remasked nodes, and the reconstructed feature matrix is ​​obtained.

[0027] The structural reconstruction loss is expressed as:

[0028]

[0029] Among them, Ai j The value of an element in the adjacency matrix of the initial graph of the citation network is represented by N, where N is the number of nodes in the citation network. This represents the value of an element in the reconstructed adjacency matrix;

[0030] The feature reconstruction loss is expressed as:

[0031]

[0032] Where, x i Z represents the original feature of node i in the citation network. i Let γ represent the features of node i after feature reconstruction, and let γ represent the scaling factor. T represents the set of nodes obtained by sampling nodes in the citation network using a random sampling strategy, and T represents the transpose.

[0033] Optionally, the loss function is expressed as:

[0034] Loss = L X +L S +εL clu ;

[0035] Where Loss represents the value of the loss function, LX represents the feature reconstruction loss, and L S L represents the structural reconfiguration loss. clu Let represent the clustering loss, and ε represent the first hyperparameter.

[0036] Optionally, the fused graph representation information is expressed as: C = [(1-α)*Z + α*H];

[0037] Where C represents the fused graph representation information, α represents the second hyperparameter, Z represents the graph structure representation information, and H represents the graph attribute representation information.

[0038] This invention also discloses a community detection system based on a graph dual autoencoder, comprising:

[0039] The graph information representation module is used to input a given citation network into a graph dual autoencoder to obtain graph structure representation information and graph attribute representation information;

[0040] The information fusion module is used to fuse graph structure representation information and graph attribute representation information to obtain fused graph representation information;

[0041] The clustering module is used to divide the fused graph representation information into communities using clustering methods to obtain community discovery results.

[0042] The present invention also discloses an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the community detection method based on the graph dual autoencoder.

[0043] The present invention also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the community discovery method based on a graph dual autoencoder.

[0044] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0045] This invention fuses graph structure representation information and graph attribute representation information, and performs community partitioning based on the fused graph representation information, thereby fully mining graph attribute information and structural information, improving the effectiveness of community partitioning in citation networks, and optimizing the exploration of citation network structure. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a schematic diagram of a community detection method based on a graph dual autoencoder provided in an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of community discovery provided in an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of a graph autoencoder structure using feature reconstruction provided in an embodiment of the present invention;

[0050] Figure 4 This is a simplified diagram of the MLP model provided in the embodiments of the present invention;

[0051] Figure 5 This is a schematic diagram of reconstruction loss calculation provided in an embodiment of the present invention;

[0052] Figure 6 This is a detailed structural schematic diagram of the dual self-encoder provided in an embodiment of the present invention;

[0053] Figure 7 This is a schematic diagram of the K-Means algorithm process provided in an embodiment of the present invention;

[0054] Figure 8 This is a schematic diagram of the loss of the dual autoencoder provided in an embodiment of the present invention;

[0055] Figure 9 This is an example diagram of the diagram structure provided in the embodiments of the present invention;

[0056] Figure 10 This is a schematic diagram of an example message passing process in the graph structure provided in the embodiments of the present invention;

[0057] Figure 11 This is a simplified diagram of an automatic encoder provided in an embodiment of the present invention;

[0058] Figure 12 This is a schematic diagram of the results of a community detection system based on a graph dual autoencoder, provided in an embodiment of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] The purpose of this invention is to provide a community discovery method, system, device, and medium based on a graph dual autoencoder, which improves the accuracy of community segmentation in citation networks.

[0061] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] like Figure 2 As shown in the diagram, the community discovery illustration provided by the embodiments of the present invention vividly explains how the present invention explores the structure of the content subject—the citation network—which is crucial to the citation analysis process. In order to make this process more in line with the information represented by the graph itself and further improve the quality of partitioning, the traditional community discovery method of solely using graph topology or node attribute information for community exploration is abandoned. Following the wave of deep learning, the invention fully utilizes the tool of graph autoencoder to study community discovery methods. Figure 2 This demonstrates the process of community detection using a graph structure, which aims to cluster graph nodes with relevant information. Figure 2 The nodes were clustered into two communities, C1 and C2, based on their professions. Community C1 includes nodes 1, 2, 3, and 4, while community C2 includes nodes 5, 6, and 7. If this structure is a citation network, articles with relevant citations can be aggregated into the same community.

[0063] In recent years, most community detection methods based on graph autoencoders have adopted a structure reconstruction approach, while a small number have adopted a node feature reconstruction approach, such as... Figure 3 As shown in the schematic diagram of the general architecture of the graph autoencoder with feature reconstruction provided in this embodiment of the invention, the operating mechanism of the traditional encoder is described. The traditional graph autoencoder with reconstructed features simply relies on minimizing the error between the input and the reconstructed signal to obtain the hidden layer feature representation of the input. This training strategy cannot guarantee the extraction of the essential features of the data. Simply relying on minimizing the reconstruction error may result in the encoder learning features that are merely copies of the original input, which makes the learning of graph representation information less than ideal. In addition, this feature reconstruction without damage may lead to unreliable architecture, further causing the designed model to have weak robustness.

[0064] When designing autoencoders, the choice of decoder generally depends on the semantic level of the target X. The more semantic information the target X contains (e.g., X is a one-hot matrix), the simpler the decoder model can be, such as a multilayer perceptron (MLP); the less semantic information the target X contains, the more complex the decoder needs to be. In recent years, graph autoencoders for reconstructing features have generally chosen simple MLPs as decoders, such as... Figure 4 As shown in the diagram, the simplified MLP model provided in this embodiment vividly illustrates the structure of a multilayer perceptron. The multilayer perceptron introduces one or more hidden layers on top of a single-layer neural network. These hidden layers are located between the input and output layers. Figure 4 In this process, the decoder reconstructs multi-dimensional node features with relatively little semantic information. The multilayer perceptron has poor performance and cannot bridge the gap between the encoder representation and the decoder target. As a result, the learned hidden layer representation H tends to be almost identical to the input feature X, which is not conducive to subsequent community partitioning.

[0065] like Figure 5 As shown in the diagram, the reconstruction loss calculation illustration provided in this embodiment of the invention demonstrates a simple reconstruction process and the main body of loss calculation. Currently, graph autoencoders with feature reconstruction often use mean squared error (MSE) for reconstruction loss calculation. Since node features are multidimensional and continuous, using traditional MSE as a standard for judging the quality of feature reconstruction is not appropriate. Specifically, experiments have shown that MSE loss can be minimized to near zero, which is insufficient for feature reconstruction; furthermore, MSE suffers from sensitivity and low selectivity. Sensitivity refers to the MSE's sensitivity to vector norms and dimensionality; extreme values ​​in certain feature dimensions can lead to overfitting by the MSE. Low selectivity indicates that the MSE's dissociative power is insufficient, preventing the model from focusing on more difficult-to-learn samples.

[0066] To address the problems existing in these community detection methods, this invention designs a graph dual autoencoder that incorporates structural reconstruction and feature reconstruction with a masking strategy for community detection. This fully utilizes the graph's topological and attribute information, and improves the model's imprecise hidden layer vector learning and weak model stability by damaging node features before reconstruction. Furthermore, in designing the graph autoencoder for reconstructing features, the invention fully considers the shortcomings of the multilayer perceptron as a decoder and the inappropriateness of using mean squared error as a standard for feature reconstruction quality, further refining the model.

[0067] Example 1

[0068] like Figure 1This embodiment provides a community detection method based on a graph dual autoencoder, including the following steps.

[0069] Step 101: Input the given citation network into the graph dual autoencoder to obtain graph structure representation information and graph attribute representation information.

[0070] The given citation network is constructed based on a dataset to be discovered by the community.

[0071] The graph dual autoencoder in step 101 is the trained graph dual autoencoder. The structure of the graph dual autoencoder of this invention is as follows: Figure 6 As shown.

[0072] Given global information about a citation network, let G = (V, A, X) denote the graph of the given citation network, where V is the set of nodes, N = |V| is the number of nodes, and A ∈ {0, 1}. N×N Let A be the adjacency matrix of the graph. ij Let X ∈ R be the value of an element in the adjacency matrix. N×d It is the feature matrix of the nodes in the graph, where x i Let N represent the i-th sample, N be the number of samples, and d represent the first feature dimension.

[0073] The graph dual autoencoder includes a first encoder and a second encoder. The first encoder is the encoder of a graph autoencoder for reconstructing the structure, and the second encoder is the encoder of a graph autoencoder for reconstructing the features. The first encoder uses a graph attention network (GAT); the second encoder uses a graph neural network (GNN).

[0074] The first encoder is used to output the features of each node after fusing neighborhood information based on the attention coefficients of each node and its neighboring nodes and the node features of the neighboring nodes, so as to obtain graph structure representation information; the nodes are nodes in the citation network.

[0075] The specific workflow of the first encoder includes:

[0076] For any node i, N i Let i be the set of its neighbors on the graph. Calculate the neighbors of node i (j∈N) one by one. i The correlation coefficient e between node i and node i ij :

[0077] e ij =a([Wx i ||Wx j ]), j∈N i ;

[0078] Where, x i For node features, the correlation between nodes i and j is accomplished through learnable parameters W and mapping function a(·): First, a linear mapping with shared parameters W increases the dimensionality of the node features, which is a common feature augmentation method; [·||·] concatenates the transformed features of nodes i and j; finally, a(·) maps the concatenated high-dimensional features to a real number.

[0079] For the correlation coefficient e ij Normalization was performed to obtain the attention coefficient α. ij :

[0080]

[0081] Based on the attention coefficient α ij The features are weighted and aggregated to obtain new features for each vertex i (incorporating neighborhood information).

[0082]

[0083] Where σ(·) is the activation function; let Z = [z′ i Z is the hidden layer representation finally obtained by the graph autoencoder for reconstructing the structure. Z is the new feature of all nodes output by GAT, that is, the graph structure representation information.

[0084] The second encoder is used to sample nodes in the citation network using a random sampling strategy to obtain a sample set. The features of the nodes in the sample set are masked using a first mask token. Graph information learning is performed on the nodes after the first mask token masking process and the nodes after the first mask token masking process, to obtain graph attribute representation information.

[0085] The specific workflow of the second encoder includes:

[0086] A uniform random sampling strategy is adopted to sample the set of nodes to obtain a sample set. Then, the features of each node in the sample set are masked using a first mask token (mask marker [MASK]), as expressed by the following formula:

[0087]

[0088] in, x is the sample set obtained by sampling nodes. [M] ∈R d As a learnable vector, x i For v i Node features ∈V The node characteristics of node i in node set V after masking and labeling.

[0089] In the graph autoencoder for reconstructing features, graph information learning (encoding learning) is performed. Taking Graph Convolutional Networks (GCNs) as an example, the encoding learning process includes:

[0090]

[0091]

[0092] Where L represents the number of layers in the GCN, H (1) H (l-1) H (l) These represent the graph information learned at layers 1, (l-1), and 1 of the GCN, respectively. The input to the first layer of the GCN is X′, which is obtained after masking and marking. Composition; φ(·) is the activation function of the fully connected layer, such as the ReLU or Sigmoid function; Let I be the degree matrix, and let W be the identity diagonal matrix of the adjacent matrix A of each node in its self-loop. (1) W (l-1) These are the weight matrices for the first and (l-1)th layers of the GCN, respectively.

[0093] The H generated in the previous step is processed using softmax normalization. (L) This yields the final hidden layer representation from the encoder stage. The last layer of the GCN module is a multi-classification layer with softmax functionality. H∈RN×dh represents the hidden layer representation obtained by the GCN encoder, which is the product of the graph autoencoder that reconstructs the features. h H represents the second feature dimension, described as:

[0094]

[0095] Among them, H (L) W is the representation learned by the Lth layer of GCN. (L) Let be the weight matrix of the Lth layer of the GCN.

[0096] Step 102: Merge the graph structure representation information and the graph attribute representation information to obtain the merged graph representation information.

[0097] The fused graph representation is: C = [(1-α)*Z + α*H];

[0098] Where C represents the fused graph representation information, α represents the second hyperparameter, Z represents the graph structure representation information, and H represents the graph attribute representation information.

[0099] Step 103: Use clustering methods to divide the fused graph representation information into communities to obtain community discovery results.

[0100] The K-Means clustering method was used.

[0101] like Figure 7 As shown in the diagram, the K-Means algorithm process provided in this embodiment vividly illustrates one clustering process of the K-Means algorithm. The idea of ​​the K-Means algorithm is as follows: First, randomly select K objects as initial cluster centers. Then, calculate the distance between each object and each seed cluster center. Then, assign each object to the cluster center closest to it. Once an object is assigned, it forms a new cluster with the cluster center, and this assignment cannot be changed. Then, recalculate the distance between each cluster center and the object, and repeat this calculation until no object can be reassigned to a different cluster, or the cluster centers no longer change, or the sum of squared errors is locally minimized. At this point, the loop calculation stops. Figure 7 In the diagram, (a) represents the original seed assignment, and the cross symbols in (b) to (f) represent the change process of cluster centers. The operating mechanism of the K-Means algorithm is as follows:

[0102] Input: Number of clusters K, iteration termination value Z

[0103] Output: Clustering results

[0104] 1: For(t=1; t<=Z; t+=1){

[0105] 2: Given a data object X i ; / / Data X i More than K

[0106] 3: Calculate the distance dist(X) between the cluster centers and the objects. i (Center);

[0107] 4: X i Drift to distance X i The cluster in which the nearest cluster center is located;

[0108] 5: For (until X) i (Cannot be allocated){

[0109] 6: Update all cluster centers ；

[0110] 7:}

[0111] 8: Output the clustering results;

[0112] 9:}

[0113] The community detection method based on graph dual autoencoders further includes training the graph dual autoencoder; the loss function used to train the graph dual autoencoder includes structural reconstruction loss, feature reconstruction loss, and clustering loss, such as... Figure 8 As shown.

[0114] The graph dual autoencoder further includes a first decoder and a second decoder. The first decoder is a decoder for a graph autoencoder that reconstructs structures, and the second decoder is a decoder for a graph autoencoder that reconstructs features.

[0115] The first decoder is used to perform an inner product operation on the graph structure representation information to obtain the reconstructed adjacency matrix.

[0116] The reconstructed adjacency matrix is ​​represented as:

[0117] in, For the reconstructed adjacency matrix, The original adjacency matrix A at the corresponding position ij Calculate the structural remodeling loss together.

[0118] The second decoder is used to: remask the nodes that have been masked by the first mask token using the second mask token; for the remasked nodes, based on the neighboring nodes of the remasked nodes, use a graph neural network to reconstruct the features of the remasked nodes to obtain the reconstructed feature matrix.

[0119] The specific workflow of the second decoder includes: masking the nodes that have been masked by the first mask token using another mask token [DMASK] (second mask token); and setting the feature vectors of the nodes corresponding to the nodes masked by the first mask token in the output H of the graph autoencoder of the reconstructed features to zero. This is the vector after masking by the second decoder. for The remasking code in the middle.

[0120]

[0121] like Figure 9 and Figure 10 As shown in the example diagram and example message passing process diagram provided in the embodiments of the present invention, the process of a node listening to information from its neighbors in a simple graph structure, then updating the information and passing it forward is called message passing.

[0122] A second decoder is used to reconstruct the input features of masked nodes from their neighboring unmasked latent representations. This reconstruction process is performed according to message propagation. The second decoder is only used during the self-supervised training phase to perform the node feature reconstruction task; therefore, the second decoder architecture is independent of encoder selection and can use any type of GNN. Given f E As a graph encoder, f D As a graph decoder, the entire learning process of the graph autoencoder for reconstructing features can be represented as:

[0123] H = f E (A, X), G′=f D (A, H).

[0124] Where G′ represents the reconstructed graph, which is the product generated by the second decoder.

[0125] This invention proposes a community detection method based on citation analysis using a graph dual autoencoder. After designing the graph dual autoencoder model, a reasonable objective function needs to be selected to optimize the model. By designing such an objective function, the quality of community partitioning can be improved, thereby promoting the community detection process.

[0126] For a citation network, communities can be segmented based on different association methods of documents, such as co-authorship, co-citation, and coupling. For example, authors who co-author a book are clustered into the same community. On the one hand, this process increases the dissemination and sharing effectiveness of scientific data. Community segmentation can uncover more documents related to a particular research direction, and this data recommendation effect facilitates acquiring more comprehensive knowledge in that research area. On the other hand, community segmentation allows for a more convenient understanding of the flow of scientific knowledge within the same community, which is beneficial for exploring the development and evolution of a particular research area.

[0127] The loss function of the graph dual autoencoder consists of reconstruction loss and clustering loss. The reconstruction loss is composed of feature reconstruction loss and structural reconstruction loss. The purpose of training the graph dual autoencoder is to minimize the reconstruction error between the input and output. The quality of the vectors ultimately learned by the graph dual autoencoder determines the model's performance.

[0128] To enhance the robustness of the model, this invention uses cosine error (SCE) instead of mean square error (MSE) to calculate the reconstruction loss after feature reconstruction.

[0129] Using cosine error as the standard for reconstructing the original node features can get rid of the influence of dimension and vector norm. The l2 norm in cosine error can map the vector to the unit hypersphere, which can greatly improve the training stability of representation learning.

[0130] The feature reconstruction loss is expressed as:

[0131]

[0132] Where, x i Z represents the original feature of node i in the citation network. i Let γ represent the features of node i after feature reconstruction, and let γ represent the scaling factor. T represents the set of nodes obtained by sampling nodes in the citation network using a random sampling strategy, and T represents the transpose.

[0133] The loss after structural reconstruction is calculated using the classic cross-entropy function. Cross-entropy measures the difference between two different probability distributions of the same random variable; in machine learning, it represents the difference between the true probability distribution and the predicted probability distribution. The smaller the cross-entropy value, the better the model's prediction performance.

[0134] The structural reconstruction loss is expressed as:

[0135]

[0136] Among them, A ij The value of an element in the adjacency matrix of the initial graph of the citation network is represented by N, where N is the number of nodes in the citation network. This represents the value of an element in the reconstructed adjacency matrix.

[0137] During feature reconstruction and structural reconstruction, the model generates corresponding graph representation information. When reconstructing features, it is desirable for the extracted features to better reflect the characteristics of the original input, thus obtaining a graph representation in which attribute information is better learned, namely the vector H output by the graph autoencoder of reconstructed features. When reconstructing structures, it is desirable for the extracted structural information to better reflect the structural characteristics of the original input, thus obtaining a graph representation in which structural information is better learned, namely the vector Z output by the graph autoencoder of reconstructed structures.

[0138] Based on the clustering results obtained in step 103, the KL divergence loss between the clustering result distribution Q and the target distribution P is calculated. The smaller the KL divergence, the closer the distributions of P and Q are. Q can be repeatedly trained to approximate the distribution of P. The clustering loss is expressed as:

[0139]

[0140] Where, q ij p represents the elements in the clustering result distribution Q. ij This represents an element in the target distribution P.

[0141]

[0142]

[0143] q ij This can be viewed as the probability of assigning node i to cluster j; f j For the i-th row q ij The sum of; h i It is the i-th row of H; μ j The classes are initialized using K-Means on the representation learned by the autoencoder before training; t is the degree of freedom of the student t-distribution.

[0144] The loss function is expressed as:

[0145] Loss = L X +L S +εL clu ;

[0146] Where Loss represents the value of the loss function, L X L represents the feature reconstruction loss. S L represents the structural reconfiguration loss. clu Let represent the clustering loss, and ε represent the first hyperparameter. ε > 0. ε is a hyperparameter that balances clustering optimization of the original data with preservation of local structure.

[0147] like Figure 11 As shown in the diagram, the simplified diagram of the graph autoencoder model provided in this embodiment of the invention simply describes the structure of the autoencoder. An autoencoder is an unsupervised application that uses backpropagation to update parameters. Its ultimate goal is to make the output x′ infinitely close to the input x. In this process, the autoencoder first compresses the input data into a lower-dimensional feature, and then uses this lower-dimensional feature to reconstruct the input data. The reconstructed data is the output of the autoencoder. Essentially, an autoencoder is a compression algorithm. An autoencoder consists of three parts: an encoder (used for data compression), a compressed feature vector (the features compressed by the encoder), and a decoder (used for data decoding). Figure 11 In the middle, W enc W represents the weight matrix of the encoder. dec This represents the weight matrix of the decoder.

[0148] This invention uses the loss function as the objective function and utilizes the objective function to fine-tune the graph dual autoencoder.

[0149] By minimizing the objective function, stochastic gradient descent (SGD) backpropagation helps the graph dual autoencoder model learn parameters that yield better clustering results, thereby improving training efficiency.

[0150] Meanwhile, weighted summation of graph representations with different emphases generated by two graph autoencoders to generate new graph representations, followed by clustering, will make the community segmentation results more accurate and of higher quality. This will help to explore more precise citation network structures and promote the citation analysis process.

[0151] To demonstrate the advancements of this invention, experiments were conducted on the real-world dataset Citeseer. The Citeseer dataset contains citation links between 3312 and 4732 papers, all belonging to six different academic research fields. Each paper is represented by a keyword derived from a 3703-dimensional word vector. Hyperparameters α and ε were set to 0.5 and 0.001, respectively. Three commonly used metrics in community detection—Accuracy of Accuracy (ACC), Normalized Mutual Information (NMI), and Adjusted Land Coefficient (ARI)—were selected to evaluate the model's performance. Based on this, comparisons with other community detection methods were made, detailed below:

[0152] K-Means (Krishna & Murty, 1999): It initializes K distinct clusters, calculates the center of each cluster using the mean calculation method, and then iteratively updates the cluster centers until the criterion function converges.

[0153] TADW (Yang et al., 2015): It integrates node text information into network representation learning through matrix factorization, thereby combining rich topological and semantic information.

[0154] GAE & VGAE (Kipf & Welling, 2016): They used a graph autoencoder constructed from graph convolutional networks to integrate topological and attribute information values ​​into the learned representation.

[0155] GraphEncoder (Salehi & Davulcu, 2020): It learns the nonlinear representation of the original network through stacked autoencoders and achieves clustering results through the K-Means method.

[0156] The experimental results are detailed in Table 1, with the best values ​​indicated in bold. A, X, and A&X represent whether the method utilizes only network topology and attribute information, or whether it utilizes both network topology and attribute information, respectively.

[0157] Table 1 shows the performance of the dual autoencoder and other community detection algorithms in the community detection task.

[0158]

[0159]

[0160] Comparison with existing data reveals that the model of this invention outperforms other methods on the Citeseer dataset. Specifically, compared to other community detection algorithms that utilize both network topology and attribute information, our model improves the accuracy (ACC), normalized mutual information (NMI), and adjusted Rand index (ARI) evaluation metrics by an average of 20.4%, 27.1%, and 21.4%, respectively, on this dataset. Compared to the traditional clustering algorithm K-Means, which only utilizes attribute information, the model of this invention improves the ACC, NMI, and ARI evaluation metrics by 9.4%, 17.8%, and 10.5%, respectively, on this dataset. Compared to the GraphEncoder algorithm from 2020, which only utilizes network topology, the model of this invention improves the ACC, NMI, and ARI evaluation metrics by 34.5%, 43.3%, and 34.7%, respectively, on this dataset, thus validating the effectiveness of this invention.

[0161] This invention promotes citation analysis by optimizing community segmentation algorithms, which helps researchers discover important literature and explore the flow of scientific knowledge in practice. The community discovery method based on a graph dual autoencoder described in this invention can be embedded in literature search or reading software. By linking a currently searched or read document to other documents with citation relationships, it helps researchers discover important literature and explore the flow of knowledge in a field (analogous to a webpage recommending other related movies when searching for or watching a particular film). A citation network consists of nodes and edges. A citation network is graph data with documents as nodes and citation links as edges. The Citeseer dataset is a citation network, but there are other citation networks besides Citeseer. This embodiment is an example to illustrate the technical effects of this invention, and the optimization of community discovery in citation networks is verified through feedback from measurement metrics.

[0162] This invention, based on citation analysis, designs a graph dual autoencoder community detection algorithm with high community partitioning quality. It proposes a feasible solution to the problem of negatively impacting the development of graph autoencoders, compensates for the deficiency of insufficient exploration of attribute information, enhances the robustness of autoencoders, thereby improving community partitioning performance and optimizing the exploration of citation network structures. At the same time, the introduction of masking concepts into the field of community detection will provide a new approach for future community detection method research, further promoting the development of citation analysis technology.

[0163] This invention proposes a community detection method based on a graph dual autoencoder, offering a feasible solution to the problems negatively impacting the development of graph autoencoders: 1. Most targets in a graph are feature vectors with limited information. Multilayer perceptrons (MLPs), commonly used as decoders in graph autoencoders (GAEs), may fail to bridge the gap between the encoder representation and the decoder target, resulting in poor graph feature acquisition. To address this, this invention proposes using a more expressive graph neural network (GNN) as the decoder. This improvement leads to better hidden layer representations in the graph autoencoder, enhancing the subsequent community partitioning performance. 2. Currently, the mean squared error (MSE) used in feature reconstruction loss calculations in graph autoencoders is affected by the existence of different feature vector norms and dimensions, posing a risk of model instability. To address this, this invention proposes using cosine error (SCE) for feature reconstruction loss calculation, enhancing the model's robustness.

[0164] This invention addresses the problem that traditional community detection methods cannot simultaneously consider network topology and node attribute features. It designs a graph dual autoencoder model that reconstructs both structure and features, utilizing both graph structural and attribute information. Furthermore, it solves the problem of insufficient learning of graph representation in recent graph autoencoder-based community detection methods that rely on a single structure or feature reconstruction approach. This invention obtains a hidden layer representation with better learned structural information through structure reconstruction and a hidden layer representation with better learned attribute information through feature reconstruction. By weighted summing of these two hidden layer representations and then supervising learning through minimizing the model's objective function, optimal graph representation information can be obtained. However, these feature-reconstructing graph autoencoders use a common architecture, which is a non-destructive feature reconstruction. Experiments have shown that this architecture may be unreliable, resulting in models with weak robustness. To address this issue, inspired by the widely used denoising autoencoder approach in computer vision and natural language processing—which destroys the input and then reconstructs it—this invention employs a reconstruction method that first destroys some of the original features of nodes before feature reconstruction. This method enhances the robustness of the graph autoencoder.

[0165] This invention fills a technological gap in the domestic and international industries: It designs a degraded feature reconstruction method based on the idea of ​​denoising autoencoders in computer vision and natural language processing that first destroy the input and then reconstruct it. Compared to traditional encoders, the hidden layer representation of the graph autoencoder using this feature reconstruction method is not directly mapped from the original input, but rather from a "corrupted" version of the original input. The encoder randomly sets a certain proportion of nodes in the original input to zero, leaving the remaining nodes unprocessed, thus obtaining a "corrupted" version of the original input. This is equivalent to introducing a certain proportion of "blank" elements into the original input, reducing the information contained within it. Then, through learning from the input and its components, it attempts to fill in the lost information, thereby learning the data structure and making the extracted features more reflective of the characteristics of the original input. By injecting noise into the input and then using the noisy "corrupted" samples to reconstruct the noise-free "clean" input, it helps to form a higher-level feature representation of the input. Traditional autoencoders rely solely on minimizing the error between the input and the reconstructed signal to obtain the hidden layer feature representation of the input. This training strategy cannot guarantee the extraction of the essential features of the data. Simply minimizing the reconstruction error may result in the encoder learning features that are merely copies of the original input. Defective feature reconstruction can avoid this problem. This invention is the first to apply this masking denoising concept to a community detection method, filling a technological gap in the industry both domestically and internationally. This innovative exploration of community detection methods further facilitates the exploration of citation network structures and will promote the development of citation analysis technology.

[0166] Example 2

[0167] like Figure 12 As shown, a community detection system based on a graph dual autoencoder includes the following structure.

[0168] Graph information representation module 201 is used to input a given citation network into a graph dual autoencoder to obtain graph structure representation information and graph attribute representation information.

[0169] The information fusion module 202 is used to fuse graph structure representation information and graph attribute representation information to obtain fused graph representation information.

[0170] Clustering module 203 is used to divide the fused graph representation information into communities using clustering methods to obtain community discovery results.

[0171] Example 3

[0172] This invention provides an electronic device including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the community detection method based on a graph dual autoencoder as described in Embodiment 1.

[0173] Alternatively, the aforementioned electronic device may be a server.

[0174] In addition, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the community discovery method based on a graph dual autoencoder of embodiment 1.

[0175] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0176] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A community discovery method based on graph double autoencoder, characterized in that, The method comprises the following steps: inputting a given citation network into a graph dual autoencoder to obtain graph structure representation information and graph attribute representation information; the graph comprises nodes and edges, and the citation network is a graph data in which documents are nodes and citation links are edges; fusing the graph structure representation information and the graph attribute representation information to obtain fused graph representation information; performing community division on the fused graph representation information by using a clustering method to obtain a community discovery result; the graph dual autoencoder comprises a first encoder and a second encoder; the first encoder is configured to output a feature of each node after fusing neighborhood information according to an attention coefficient of each node and neighbor nodes and node features of the neighbor nodes to obtain the graph structure representation information; the nodes are nodes in the citation network; the second encoder is configured to sample the nodes in the citation network by using a random sampling strategy to obtain a sampling set, mask features of the nodes in the sampling set by using a first mask token, and learn graph information of the nodes masked by using the first mask token and the nodes not masked by using the first mask token to obtain the graph attribute representation information; the community discovery method based on the graph dual autoencoder further comprises training the graph dual autoencoder; and a loss in a loss function used for training the graph dual autoencoder comprises a structure reconstruction loss, a feature reconstruction loss and a clustering loss; the graph dual autoencoder further comprises a first decoder and a second decoder; the first decoder is a decoder of a graph autoencoder for reconstructing a structure; and the second decoder is a decoder of a graph autoencoder for reconstructing a feature; the first decoder is configured to perform an inner product operation on the graph structure representation information to obtain a reconstructed adjacency matrix; the second decoder is configured to: re-mask the nodes masked by using the first mask token by using a second mask token; for the nodes after the re-masking, reconstruct features of the nodes after the re-masking based on neighbor nodes of the nodes after the re-masking by using a graph neural network to obtain a reconstructed feature matrix; the structure reconstruction loss is represented as: ； wherein, denotes the value of an element in the adjacency matrix of the initial graph of the citation network, N is the number of nodes in the citation network, denotes the value of an element in the reconstructed adjacency matrix; the feature reconstruction loss is represented as: ； wherein, denotes the original features of the nodes in the citation network, i denotes the features of the nodes after feature reconstruction, i denotes a scaling factor, denotes a set of nodes obtained by sampling the nodes in the citation network using a random sampling strategy, and T denotes a transpose.​​ 2.The community discovery method based on graph dual auto-encoder according to claim 1, characterized in that, the graph autoencoder for reconstructing the structure adopts a graph attention network; and the graph autoencoder for reconstructing the feature adopts a graph neural network. 3.The community discovery method based on graph dual auto-encoder according to claim 1, characterized in that, the loss function is represented as: ； wherein, denotes a value of the loss function, denotes a feature reconstruction loss, denotes a structure reconstruction loss, denotes a clustering loss, denotes a first hyperparameter. 4.The community discovery method based on graph dual auto-encoder according to claim 1, characterized in that, The fused graph representation information is represented as: ; wherein, represents the fused graph representation information, represents the second hyperparameters, represents the graph structure representation information, represents the graph property representation information.

5. A community discovery system based on graph dual autoencoder, characterized in that, The method comprises the following steps: a graph information representation module is configured to input a given citation network into a graph dual autoencoder to obtain graph structure representation information and graph attribute representation information; the graph comprises nodes and edges, and the citation network is a graph data in which documents are nodes and citation links are edges; an information fusion module is configured to fuse the graph structure representation information and the graph attribute representation information to obtain fused graph representation information; a clustering module is configured to perform community division on the fused graph representation information by using a clustering method to obtain a community discovery result; the community discovery method based on the graph dual autoencoder further comprises training the graph dual autoencoder; and a loss in a loss function used for training the graph dual autoencoder comprises a structure reconstruction loss, a feature reconstruction loss and a clustering loss; The graph dual autoencoder further comprises a first decoder and a second decoder, the first decoder being a decoder of the structure-reconstruction graph autoencoder, and the second decoder being a decoder of the feature-reconstruction graph autoencoder; The first decoder is configured to perform an inner product operation on the graph structure representation information to obtain a reconstructed adjacency matrix; The second decoder is configured to: re-mask the processed nodes using the first mask token with a second mask token; for the re-masked nodes, reconstruct features of the re-masked nodes based on neighbor nodes of the re-masked nodes using a graph neural network to obtain a reconstructed feature matrix; The structure reconstruction loss is represented as: ； wherein, denotes the value of an element in the adjacency matrix of the initial graph of the citation network, N is the number of nodes in the citation network, denotes the value of an element in the reconstructed adjacency matrix; The feature reconstruction loss is represented as: ； wherein, denotes the original features of the nodes in the citation network, i denotes the features of the nodes after feature reconstruction, i denotes a scaling factor, denotes a set of nodes obtained by sampling the nodes in the citation network using a random sampling strategy, and T denotes a transpose.​​ 6. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory is configured to store a computer program, and the processor is configured to run the computer program to enable the electronic device to perform the community discovery method based on the graph dual autoencoder according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer program stored in the memory is executed by the processor to implement the community discovery method based on the graph dual autoencoder according to any one of claims 1 to 4.

Images