Academic paper clustering method and system based on cluster feature learning

Abstract

The application relates to the technical field of academic paper analysis based on machine learning, in particular to an academic paper clustering method and system based on cluster feature learning. A citation network or a collaboration network object of academic papers to be clustered is represented as an attribute graph G, a node is used to represent an academic paper, an edge is used to represent a citation relationship between the academic papers or a collaboration relationship between paper authors, an adjacent matrix is formed by the edge relationships between all academic paper nodes, and a node attribute matrix is formed by academic paper node title feature vectors; a graph encoder is constructed to encode the nodes of the attribute graph G into a feature space, the graph encoder is iteratively trained through a group discriminant loss function, and cluster features of node embedding in the attribute graph G are learned and acquired; and the node embedding containing the cluster feature information in the attribute graph G is subjected to clustering analysis based on a K-means algorithm, so that a clustering division result of the academic paper object to be clustered is acquired. The application can improve and strengthen the paper node cluster feature learning capability, so that the clustering effect of the paper nodes is improved.

Description

Technical Field

[0001] This invention relates to the field of academic paper analysis technology based on machine learning, and in particular to an academic paper clustering method and system based on cluster feature learning. Background Technology

[0002] Academic paper analysis is the process of analyzing citation relationships (citation networks) between academic papers and collaborative relationships (collaboration networks) among their authors. These networks can reveal the structure and dynamics of a scientific field, such as research areas, influential papers, emerging topics, and collaboration patterns. However, analyzing these networks is challenging due to their large scale, complexity, and heterogeneity. Therefore, effective and efficient techniques are needed to cluster these networks into meaningful groups based on their structural and attribute characteristics.

[0003] Deep graph clustering is one of the main methods for analyzing academic papers. It aims to divide entities into different clusters based on similarity in an unsupervised manner, making it an important method for analyzing data with unknown categories. Over the past few decades, numerous shallow clustering algorithms have emerged, including centroid-based, density-based, distribution-based, hierarchical, ensemble, and multi-view clustering. While these algorithms have achieved good clustering results, they are highly dependent on the features of the input data; different datasets typically require different similarity measures and separation techniques. Furthermore, due to a lack of feature learning capabilities, these clustering algorithms often have limited performance on complex data. To address the feature learning problem, much clustering-related work utilizes dimensionality reduction and representation learning algorithms for feature extraction or transformation, mapping the original complex data to a feature space that more easily separates different clusters before clustering. Typical dimensionality reduction and representation learning algorithms include principal component analysis, kernel methods, spectral decomposition, and deep neural networks. Thanks to the rapid development of deep learning technology, deep learning-based clustering methods (referred to as deep clustering) have performed exceptionally well and have been widely applied and achieved significant success in structured data such as text, images, and audio / video. Compared to structured data, graph data is widely used to describe entities and their interactions in the real world. It contains nodes and edges connecting them, exhibiting more complex and irregular unstructured characteristics, making it difficult to directly apply traditional deep learning techniques. As a deep learning framework that directly learns from graph data, Graph Neural Networks (GNNs) successfully map graph data to a low-dimensional vector space by passing aggregate node information through the graph's topological structure, and is used for downstream clustering tasks. Therefore, deep graph clustering techniques based on GNNs have received widespread attention. However, the representation learning in deep graph clustering algorithms focuses on the feature representation of individual nodes, and the loss function design emphasizes learning the features of the nodes themselves, paying less attention to other cluster features such as connections and structures between nodes. It usually requires downstream clustering task design to guide the model in learning cluster features. Although current mainstream clustering models have proposed mechanisms such as pseudo-labels and self-training to increase the model's ability to learn cluster features, the inherent limitations of representation learning algorithms in capturing cluster features still restrict the application of deep graph clustering models in academic paper cluster analysis. Summary of the Invention

[0004] To address this, the present invention provides an academic paper clustering method and system based on cluster feature learning, which solves the problem of limited application of existing deep learning models in academic paper clustering analysis. It improves the clustering effect of each paper node by enhancing and strengthening the cluster feature learning ability of paper nodes.

[0005] According to the design scheme provided by this invention, on the one hand, an academic paper clustering method based on cluster feature learning is provided, the process of which includes:

[0006] The academic paper citation network or collaboration network object to be clustered is represented as an attribute graph G, where nodes represent academic papers, edges represent citation relationships between academic papers in the citation network or collaboration relationships between paper authors in the collaboration network, the edge relationships between all academic paper nodes form an adjacency matrix, and the node attribute matrix is ​​composed of the title feature vectors of each academic paper node.

[0007] A graph encoder is constructed to encode all nodes of the attribute graph G into the feature space, and the graph encoder is iteratively trained using a preset group discrimination loss function to learn and acquire the cluster features embedded in the nodes of the attribute graph G.

[0008] Cluster analysis is performed on the node embeddings containing cluster features in the attribute graph G based on the K-means algorithm to obtain the clustering results of the academic paper citation network or collaborative network objects to be clustered.

[0009] Furthermore, the graph encoder is iteratively trained, including:

[0010] In each iteration, firstly, positive and negative sample groups are constructed based on contrastive learning theory. In cluster feature learning, the node distribution between the positive and negative sample groups is widened to learn and obtain the positive sample node embedding that approximates the original node distribution, so as to perform clustering analysis based on the sample node embedding.

[0011] Furthermore, positive and negative sample groups are constructed based on contrastive learning theory, including:

[0012] The dropout operation is used to randomly mask the node title features in the node attribute matrix. The randomly masked node attribute matrix is ​​used as the current node attribute matrix. The enhanced adjacency matrix obtained in the previous iteration is used as the current adjacency matrix. An enhanced view is constructed based on the current node attribute matrix and the current adjacency matrix. The enhanced view is used as the current positive sample. The structure of the current node attribute matrix is ​​destroyed by randomly changing the order of nodes in the node attribute matrix to generate a destroyed map as the current negative sample.

[0013] Furthermore, in cluster feature learning, the node embeddings of positive samples that approximate the original node distribution are learned and obtained by widening the node distribution between positive and negative sample groups, including:

[0014] First, the node attribute matrix and adjacency matrix of the current positive and negative samples are used as inputs to the graph encoder, and the corresponding positive sample node embedding vector and negative sample node embedding vector are obtained based on the graph neural network in the graph encoder.

[0015] Next, the encoder parameters are updated by merging and aggregating the positive sample node embedding vectors and the negative sample node embedding vectors, and by minimizing the preset binary cross-entropy target loss function.

[0016] Then, the similarity between each pair of positive sample nodes is calculated based on the embedding vector of the positive sample nodes. A similarity matrix is ​​constructed based on the similarity, and the top k highly similar neighbor nodes are selected for each node to construct new connection edges. The enhanced adjacency matrix in the current iteration round is updated and obtained with the new connection edges.

[0017] Furthermore, based on the merging and aggregation of positive and negative sample node embedding vectors, the encoder parameters are updated by minimizing a preset binary cross-entropy objective loss function, including:

[0018] First, the embedding vectors of positive and negative sample nodes are concatenated and merged. Then, the sample set is obtained by summing the feature vectors of each node title. Finally, the accuracy of the positive and negative sample node discrimination is measured using a preset binary cross-entropy target loss function.

[0019] Furthermore, the presupposed binary cross-entropy objective loss function is expressed as follows: in This represents the result of merging and aggregating the embedding vectors of node i, y i The label representing node i, when node i originates from a positive sample, y i =1, otherwise y i =0, where N is the number of nodes in the positive / negative samples.

[0020] Furthermore, the pairwise similarity between positive sample nodes is calculated based on the embedding vectors of the positive sample nodes, including:

[0021] Based on the positive sample node embedding vectors, the similarity between pairwise node pairs is calculated using cosine distance.

[0022] Furthermore, for each node, the top k highly similar neighbor nodes are selected to construct new connection edges. These new connection edges are used to update and obtain the enhanced adjacency matrix in the current iteration round, which includes:

[0023] First, the KNN classification algorithm is used to retain k highly similar nodes as neighbors for each node in the similarity matrix, resulting in a relatively sparse adjacency matrix. Next, the relatively sparse adjacency matrix is ​​symmetrically normalized to maintain its symmetry. Then, a pre-defined update formula is used... To obtain the enhanced adjacency matrix in the current iteration round, where ε is the graph structure update weight hyperparameter, and A * Let A represent the adjacency matrix after symmetric normalization, and let A represent the adjacency matrix of the attribute graph G.

[0024] Furthermore, this invention also provides an academic paper clustering system based on cluster feature learning, comprising: a graph construction module, a graph encoding module, and a cluster analysis module, wherein...

[0025] The graph construction module is used to represent the academic paper citation network or collaboration network objects to be clustered as an attribute graph G. Here, nodes represent academic papers, edges represent the citation relationships between academic papers in the citation network or the collaboration relationships between paper authors in the collaboration network, the edge relationships between all academic paper nodes form an adjacency matrix, and the node attribute matrix is ​​composed of the title feature vectors of each academic paper node.

[0026] The graph encoding module is used to construct a graph encoder to encode all nodes of the attribute graph G into the feature space, and to iteratively train the graph encoder through a preset group discrimination loss function to learn and obtain the cluster features embedded in the nodes of the attribute graph G.

[0027] The clustering analysis module is used to perform clustering analysis on the node embeddings containing cluster features in the attribute graph G based on the K-means algorithm, and to obtain the clustering results of the academic paper citation network or collaborative network objects to be clustered.

[0028] The beneficial effects of this invention are:

[0029] This invention differs from previous methods that primarily focus on learning node feature representations. It trains the graph encoder based on the idea of ​​contrastive learning, and retains more cluster features, including node features, graph structure, and topology, by learning node embeddings that approximate the original distribution. This makes it more suitable for graph clustering analysis tasks in academic papers. The edges reconstructed based on cluster features can integrate the learned cluster features into the graph structure, thereby enhancing the effect of iterative training. At the same time, it can improve the original graph structure. Through better graph encoder feature representation, it can achieve better paper clustering and partitioning results, and has good application prospects. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the academic paper clustering process in the embodiment;

[0031] Figure 2 This is a schematic diagram illustrating the principle of the FCFL algorithm for clustering academic papers based on cluster feature learning in this embodiment.

[0032] Figure 3 This is a schematic diagram of the ablation experiment analysis of the cluster feature learning mechanism CL and the graph structure optimization design G on four datasets in the example;

[0033] Figure 4 This is a visualization of how the learned node embeddings are used to calculate the pairwise similarity matrix between nodes on the Cora dataset in this embodiment.

[0034] Figure 5 This example illustrates the numerical impact of updating the hyperparameter adjacency matrix weight ε and the number of nearest neighbors k on different datasets. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer and more understandable, the invention will be further described in detail below with reference to the accompanying drawings and technical solutions.

[0036] With the rise and development of contrastive learning techniques, deep graph clustering methods based on contrastive learning alleviate the problem of manual trial and error and improve clustering performance by designing a more consistent and discriminative contrastive loss function to replace the clustering guidance loss function in network training. The core idea of ​​contrastive learning is to make the representations of nodes of the same type more similar in the feature space, and to make the representations of nodes of different types more distant, thereby improving the discriminative ability of node features to better distinguish nodes of different categories. Although deep graph clustering algorithms have achieved good results, their representation learning mainly focuses on learning node features, neglecting cluster features that represent the connections between nodes of the same type. According to the characteristics of clustering tasks in academic papers, learning cluster features is key to dividing nodes into different clusters, because cluster features not only include the features of the nodes themselves, but also the connections and topological structure between nodes, which are global features and more important for clustering tasks. In the embodiments of this invention, see... Figure 1 As shown, firstly, an academic paper clustering method based on cluster feature learning is provided, which includes:

[0037] S101. Represent the academic paper citation network or collaboration network object to be clustered as an attribute graph G, where nodes represent academic papers, edges represent citation relationships between academic papers in the citation network or collaboration relationships between paper authors in the collaboration network, the edge relationships between all academic paper nodes form an adjacency matrix, and the node attribute matrix is ​​composed of the title feature vectors of each academic paper node.

[0038] The attribute graph can be represented as G=(V,ε,X)=(A,X), where V={v1,v2,…,v n} represents the set of academic paper nodes in the citation network or collaboration network object, which can be set to contain n nodes of K types, and ε represents the set of edges of each citation relationship or collaboration relationship of the academic paper nodes. This represents the node attribute matrix, where the vector x in the i-th column is... i It is node v i The title feature vector, and Let represent the adjacency matrix of the graph. The degree matrix is ​​represented as . The symbols are summarized in Table 1.

[0039] Table 1 Summary of Symbols

[0040]

[0041] S102. Construct a graph encoder to encode all nodes of the attribute graph G into the feature space, and iteratively train the graph encoder using a preset group discrimination loss function to learn and acquire the cluster features embedded in the nodes of the attribute graph G.

[0042] Cluster features represent the characteristics that distinguish one group of nodes in a graph from other groups, mainly including node attributes, edges, and topological features. Cluster features reflect the similarity between nodes within the same cluster and the differences between different clusters, and are the key basis for dividing graph data into different groups. In deep graph clustering, given a graph G, deep graph clustering aims to train a graph encoder in an unsupervised manner to encode all nodes in the graph into a low-dimensional feature space. By learning and utilizing the cluster features of the graph data, the nodes are divided into k disjoint groups {C1, C2, ... Cn}. k}, the same group C k The nodes in the model have high similarity.

[0043] In this embodiment, the iterative training of the graph encoder can be specifically designed to include the following:

[0044] In each iteration, firstly, positive and negative sample groups are constructed based on contrastive learning theory. In cluster feature learning, the node distribution between the positive and negative sample groups is widened to learn and obtain the positive sample node embedding that approximates the original node distribution, so as to perform clustering analysis based on the sample node embedding.

[0045] In this embodiment, the purpose of attribute graph clustering is to train a graph encoder in an unsupervised manner to encode all nodes in the graph into a low-dimensional feature space. By learning and utilizing the cluster features of the graph data, the nodes are divided into several disjoint groups {C1, C2, ... C}. k}, the same group C k The nodes in the model exhibit high similarity. Given encoder inputs A and X, it can output the learned representation. Where d' represents the hidden layer dimension. Finally, based on the learned representation H, the K-means algorithm is used to obtain the clustering results.

[0046] Compared to individual node features, cluster features contain more information, including node features within a cluster, structural and topological information, etc. All cluster features of a dataset combined represent the global features of the dataset. Therefore, cluster features represent richer information in graph data, and fully utilizing this information can lead to better clustering results. To learn cluster features, the model can learn a node distribution that more closely approximates the original distribution. This means that information from the original graph data, including cluster features, can be better preserved when obtaining node embeddings.

[0047] In this embodiment, a graph encoder that directly learns cluster features is trained to approximate the original node distribution, thereby preserving more cluster feature information in the node embedding. Specifically, inspired by the work of MVGRL and GGD, the idea of ​​contrastive learning can be used to construct positive and negative sample groups. By widening the distance between the node distributions of the positive and negative sample groups, a positive sample node embedding that is infinitely close to the original node distribution can be learned.

[0048] Specifically, constructing positive and negative sample groups based on contrastive learning theory can be designed to include the following:

[0049] The dropout operation is used to randomly mask the node title features in the node attribute matrix. The randomly masked node attribute matrix is ​​used as the current node attribute matrix. The enhanced adjacency matrix obtained in the previous iteration is used as the current adjacency matrix. An enhanced view is constructed based on the current node attribute matrix and the current adjacency matrix. The enhanced view is used as the current positive sample. The structure of the current node attribute matrix is ​​destroyed by randomly changing the order of nodes in the node attribute matrix to generate a destroyed map as the current negative sample.

[0050] In each iteration, the original graph structure is updated using the reconstructed adjacency matrix, and dropout is used to randomly mask the features of the nodes in the graph, resulting in a set of enhanced views. The goal is to obtain an enhanced view containing more cluster feature information, while simultaneously enhancing the model's generalization ability using feature masking. This can then be achieved by randomly changing the nodes in the attribute matrix. The destructive graph is obtained by breaking the structure of the original graph in the order specified in the graph. After continuous iteration, the destruction graph presents different distributions of negative sample nodes for comparative learning.

[0051] In cluster feature learning, positive sample node embeddings that approximate the original node distribution are learned and obtained by widening the node distribution between positive and negative sample groups. This can be designed to include the following:

[0052] First, the node attribute matrix and adjacency matrix of the current positive and negative samples are used as inputs to the graph encoder, and the corresponding positive sample node embedding vector and negative sample node embedding vector are obtained based on the graph neural network in the graph encoder.

[0053] Next, the encoder parameters are updated by merging and aggregating the positive sample node embedding vectors and the negative sample node embedding vectors, and by minimizing the preset binary cross-entropy target loss function.

[0054] Then, the similarity between each pair of positive sample nodes is calculated based on the embedding vector of the positive sample nodes. A similarity matrix is ​​constructed based on the similarity, and the top k highly similar neighbor nodes are selected for each node to construct new connection edges. The enhanced adjacency matrix in the current iteration round is updated and obtained with the new connection edges.

[0055] Using the neural network architecture in GGD, the architecture can be designed to include two graph convolutional encoders with shared weights and a multilayer perceptron. The input graph is the augmented node attribute matrix. and adjacency matrix The encoder encodes the positive sample group nodes as Then, the negative sample group nodes are encoded using the same encoder.

[0056]

[0057]

[0058] Where I represents the identity matrix of dimension n, Representing the adjacency matrix The degree matrix is ​​given by W1 and W2, which represent the learnable parameters of GCN and MLP, respectively, and σ(·) is a ReLU activation function.

[0059] The group discrimination mechanism provides a self-supervised signal by constructing an objective function to correctly distinguish between positive and negative groups of node samples. Group discrimination-based contrastive learning learns a positive sample node embedding that is infinitely close to the original node distribution by widening the node distribution between the positive and negative sample groups. This allows the node embedding to contain more cluster feature information. Therefore, in this embodiment, group discrimination-based contrastive learning is used to alleviate the problem of insufficient cluster feature learning in the model.

[0060] Specifically, the nodes of the two sets of positive and negative samples are first embedded. and Merge into a data sample set of size 2n Then, the feature vectors of each node are summed to obtain a sample set with a dimension of 2n×1. Finally, a binary cross-entropy loss function is used to measure the accuracy of node discrimination between positive and negative sample groups. The binary cross-entropy loss function can be specifically expressed as:

[0061]

[0062] in This represents the result of embedding and aggregating node i, derived from H output by the encoder. The label representing node i, when node i comes from the positive sample group, y i =1, otherwise y i=0.

[0063] Compared to text and image data, a key characteristic of graph data is its intuitive topological structure. Graph representation learning algorithms rely on this topological structure for information transmission. Learned cluster features can be integrated into the graph structure by constructing a KNN graph, thereby enhancing the learning of cluster features. In this embodiment, the KNN classification algorithm can be used to retain k highly similar nodes as neighbors for each node in the similarity matrix, resulting in a relatively sparse adjacency matrix. Next, the relatively sparse adjacency matrix is ​​symmetrically normalized to maintain its symmetry. Finally, a preset update formula is used to obtain the enhanced adjacency matrix for the current iteration.

[0064] Specifically, in augmented graph embedding Based on this, the similarity between each pair of nodes is calculated to construct a similarity matrix S. For each node, the top k highly similar neighboring nodes are selected to construct connecting edges. Finally, after symmetric normalization, a new adjacency matrix A is obtained. * Used to update the original graph A:

[0065]

[0066] Where φ(·) is a parameterless metric function, and cosine distance can be used to calculate the similarity between node pairs.

[0067] A * =q(S) (5)

[0068] q(·) indicates the post-processing procedure.

[0069]

[0070] in, This represents the updated adjacency matrix, used for iterative graph representation learning. ε is a hyperparameter representing the graph structure update weights; it is generally chosen to have a small value to maintain the stability of the original graph structure.

[0071] Post-processing involves constructing a KNN graph using the learned similarity matrix S and performing symmetric normalization. Dense similarity matrices are computationally inefficient; therefore, for each node, k highly similar nodes are retained as neighbors, resulting in a relatively sparse adjacency matrix. Finally, symmetric normalization maintains the symmetry of the adjacency matrix and ensures that edge weights are within the range [0,1].

[0072] S Top-k =Top-k(S) (7)

[0073]

[0074] Where σ q(·) is a non-linear activation function that ensures the non-negativity of the weights. In this embodiment, the ReLU function can be used.

[0075] The optimization objective of the graph encoder in this case is to minimize the group discrimination loss function L in equation (3). During training, the widely used Adam optimizer can be used to minimize L.

[0076] S103. Based on the K-means algorithm, perform cluster analysis on the node embeddings containing cluster features in the attribute graph G to obtain the clustering results of the academic paper citation network or collaborative network objects to be clustered.

[0077] See Figure 2 As shown, the specific implementation of the clustering analysis algorithm in academic papers can be divided into three main parts: a contrastive learning module that directly learns cluster features based on a group discrimination mechanism, a structure optimization module that integrates the learned cluster feature information into the original graph structure, and a clustering module. The cluster feature learning module trains an encoder to directly learn cluster features, and then the structure optimization module integrates the learned cluster features into the original graph structure to enhance cluster feature learning. In the cluster feature contrastive learning module, firstly, enhanced and corrupted graphs are designed as positive and negative sample groups, and the corresponding node embeddings are obtained using a graph encoder. Then, based on the group discrimination mechanism, the node distribution of the positive and negative sample groups is inferred, enabling the model to learn cluster features that can distinguish different sample groups. In the graph structure optimization module, based on the learned node embeddings containing cluster features, the graph adjacency matrix is ​​reconstructed based on a similarity metric function, and the original graph structure is updated to enhance the model's learning of cluster features. Finally, after iterative training in the contrastive learning module and the structure optimization module, a shallow clustering algorithm (e.g., k-means) is used to obtain the clustering results. Figure 2 In the middle, (a) the cluster feature learning module first obtains an enhanced graph through edge dropout and graph structure update enhancement techniques. The destruction diagram is obtained by randomly shuffling the node order. Then, a shared-weight GNN is used to encode the augmented and corrupted graphs to obtain positive and negative sample embeddings. Finally, model training is carried out by discriminating the node embeddings of the two groups; (b) Graph structure optimization module. Embedding the positive sample group As input, the adjacency matrix is ​​reconstructed using a similarity metric function and then symmetrically normalized to obtain a new adjacency matrix A. * (c) After iterative training of the GNN, a shallow clustering algorithm, such as k-means, is used to update and enhance the original graph structure A; Clustering is performed to divide the nodes into different groups. A specific implementation of the FCFL algorithm can be designed as follows:

[0078]

[0079] On another front, based on the above method, this embodiment of the invention also provides an academic paper clustering system based on cluster feature learning, comprising: a graph construction module, a graph encoding module, and a cluster analysis module, wherein...

[0080] The graph construction module is used to represent the academic paper citation network or collaboration network objects to be clustered as an attribute graph G. Here, nodes represent academic papers, edges represent the citation relationships between academic papers in the citation network or the collaboration relationships between paper authors in the collaboration network, the edge relationships between all academic paper nodes form an adjacency matrix, and the node attribute matrix is ​​composed of the title feature vectors of each academic paper node.

[0081] The graph encoding module is used to construct a graph encoder to encode all nodes of the attribute graph G into the feature space, and to iteratively train the graph encoder through a preset group discrimination loss function to learn and obtain the cluster features embedded in the nodes of the attribute graph G.

[0082] The clustering analysis module is used to perform clustering analysis on the node embeddings containing cluster features in the attribute graph G based on the K-means algorithm, and to obtain the clustering results of the academic paper citation network or collaborative network objects to be clustered.

[0083] To verify the effectiveness of this solution, the following explanation is based on experimental data:

[0084] Experiments were conducted on three widely used benchmark datasets: Cora, Citeseer, and PubMed. Dataset statistics are shown in Table 1. The AMAP dataset contains a large number of edges and can be considered a dense graph.

[0085] The Cora dataset is a citation network where nodes represent papers from seven different research topics in the field of machine learning, and edges represent citation relationships between papers.

[0086] The Citeseer dataset is a citation network containing six types of machine learning papers (Agent, AI, DB, IR, ML, and HCI), where nodes represent papers and edges represent citation relationships.

[0087] The ACM dataset is an author collaboration network where nodes represent papers on three different research topics: databases, wireless communications, and data mining, and edges represent the collaboration relationships between authors.

[0088] Amap is a set of photo links on the Amazon website. Nodes represent the product number codes for eight product categories, and edges represent the links between products.

[0089] Table 1 Dataset Statistics

[0090]

[0091] To measure the performance of node clustering methods, four metrics widely used in deep clustering methods were employed to evaluate the clustering results of all models: accuracy (ACC), normalized mutual information (NMI), adjusted Land coefficient (ARI), and F1 score.

[0092] Adam was used as the optimizer for model training. After training, the k-means algorithm was executed to obtain clustering results. All models were run 10 times, and the average of the results was used for comparison. The clustering results were regenerated using the model source code and parameters. The hidden layer dimensions and learning rates of the models are shown in the table below:

[0093] Table 2 Dataset Parameter Settings

[0094]

[0095] To verify the effectiveness of the model in this algorithm, comparative experiments were conducted on four benchmark datasets, comparing eight state-of-the-art deep clustering methods. Specifically, these eight methods were divided into two types: deep graph clustering algorithms based on autoencoders and deep graph clustering algorithms based on contrastive learning. The experimental results in Table 3 lead to the following two conclusions: (1) The insufficient cluster feature learning ability of representation learning models limits the performance of clustering tasks. As a graph representation learning model, GGD, without any guidance from clustering-related information, outperformed most baseline models in clustering performance on some datasets. This demonstrates the effectiveness of cluster feature learning based on group discrimination mechanisms in clustering tasks. It also shows that the neglect of cluster features in the representation learning process of existing models limits their discrimination ability. (2) Cluster feature learning is an important direction for improving the performance of clustering models. Compared with existing models, the FCFL algorithm proposed in this case performs best or second best on all datasets. This shows that the enhanced cluster feature learning mechanism proposed in this embodiment can effectively extract cluster feature information and is more suitable for clustering tasks. In conclusion, these experimental results demonstrate the superiority of the proposed solution.

[0096] Table 3 shows the average clustering performance after ten runs on four benchmark datasets, evaluated by four metrics including mean and standard deviation. Bold lines with double underlines represent the best results, while single underlines represent the next best results.

[0097]

[0098] The effectiveness of the proposed cluster feature learning mechanism CL and graph structure optimization design G in this scheme is verified through ablation experiments. Specifically, a traditional contrastive loss-trained self-supervised representation learning model, denoted as "B", is used as the baseline model in the graph. "B+CL", "B+G", and "Ours" represent the use of the cluster feature learning module, the graph structure optimization module, and both simultaneously on the baseline, respectively. Figure 3 The following results can be observed from the experimental data: (1) The cluster feature learning module CL improves the performance of the baseline because the positive sample group node distribution is moved away from the negative sample distribution that changes continuously with the iteration process through the group discrimination mechanism, thereby continuously approaching the original node distribution, so the model learns more cluster feature information; (2) The graph structure optimization design G can also improve the performance of the baseline, especially the effect is very obvious in the dense graph amap dataset. This shows the effectiveness of the graph structure optimization module and its advantage in datasets with a large number of edges, and also proves that the learned cluster features can be further strengthened by structural optimization; (3) The combination of CL and G achieves the optimal clustering performance. Overall, a large number of experiments further prove the effectiveness of the cluster feature learning mechanism CL and the graph structure optimization design G involved in the algorithm of this case.

[0099] To analyze the effectiveness of the algorithm in learning cluster features, the node embeddings learned by the model were analyzed. Calculate the pairwise similarity between nodes in the CORA dataset and visualize it. For easier comparison, sort the nodes according to their actual category, such as... Figure 4 As shown, comparing subgraphs (a) and (c) reveals that the cluster feature learning mechanism CL effectively learns the features of different clusters, making the distinction between node representations of different clusters more obvious, thus facilitating clustering tasks. Comparing the results of subgraphs (a)(b) and (c)(d) respectively, it can be seen that the graph structure optimization module G further strengthens the learning of similarity between nodes, but it also easily learns noise. Therefore, it is usually necessary to further improve the learning effect based on a high-quality representation learning model. In summary, the visualization analysis leads to the conclusion that the cluster feature learning mechanism in this algorithm effectively learns distinct cluster features, obtaining high-quality node representations. Then, the graph structure optimization mechanism further strengthens the cluster feature learning, thereby achieving better clustering results.

[0100] The effects of hyperparameter adjacency matrix update weights ε and the number of nearest neighbors k are analyzed on multiple datasets by fixing other parameters and adjusting the size of the parameters to be tested. Figure 5 The results show the impact of various parameters on clustering performance. Figure 5The following phenomena can be observed: (1) In the CORA, CITESEER, and ACM datasets, as ε increases, the clustering metric score gradually decreases. When ε is in the range of (0, 0.0001], the clustering metric score is significantly higher than when ε = 0. In the AMAP dataset, the trend of ε changes in the opposite direction. (2) In the AMAP dataset, the change of K has a more significant impact on the clustering metric, while the impact is relatively low in other datasets.

[0101] In the graph structure optimization module of the algorithm implementation in this case, ε controls the speed of updating the original graph structure, and represents K. K represents the number of reconstructed edges for each node, and the cluster information and noise information learned by ε together affect the graph structure. The analysis yields the following two conclusions: (1) Optimizing the original graph structure through graph reconstruction, while bringing beneficial cluster information and harmful noise information, and slowly updating the graph structure with small weights is a better method; (2) The graph structure optimization module has significant advantages in clustering on dense graphs. By strengthening the learning of cluster information, it is beneficial to strengthen the connection strength of edges within clusters and further improve the learning effect.

[0102] The experimental data above further demonstrates that the proposed solution achieves better clustering results than other comparative solutions in the clustering analysis of academic papers in citation networks and collaboration networks on graph data, and is more suitable for clustering and classifying academic papers in citation networks or collaboration networks.

[0103] Unless otherwise specifically stated, the relative steps, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0104] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0105] The units and method steps of the various examples described in conjunction with the embodiments disclosed in this patent can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation is not considered to be beyond the scope of this invention.

[0106] Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Optionally, all or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiments can be implemented in hardware or as a software functional module. This invention is not limited to any particular combination of hardware and software.

[0107] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for clustering academic papers based on cluster feature learning, characterized in that, Include: The academic paper citation network or collaboration network object to be clustered is represented as an attribute graph G, where nodes represent academic papers, edges represent citation relationships between academic papers in the citation network or collaboration relationships between paper authors in the collaboration network, the edge relationships between all academic paper nodes form an adjacency matrix, and the node attribute matrix is ​​composed of the title feature vectors of each academic paper node. A graph encoder is constructed to encode all nodes of the attribute graph G into the feature space. The graph encoder is then iteratively trained using a pre-defined group discriminant loss function to learn and acquire cluster features of node embeddings in the attribute graph G. These cluster features include the characteristics of the nodes themselves, as well as the relationships and topological structure between nodes. The combination of all cluster features of a dataset represents the global features of the dataset. In each iteration, positive and negative sample groups are first constructed based on contrastive learning theory. During cluster feature learning, the node distribution between the positive and negative sample groups is widened to learn and acquire positive sample node embeddings that approximate the original node distribution, enabling cluster analysis based on these sample node embeddings. In this process, the node attribute matrices and adjacency matrices of the current positive and negative samples are used as inputs to the graph encoder. The graph neural network in the graph encoder is used to obtain the corresponding positive sample node embedding vectors and negative sample node embedding vectors. The positive sample node embedding vectors and negative sample node embedding vectors are merged and aggregated, and the encoder parameters are updated by minimizing the preset binary cross-entropy objective loss function. The similarity between each pair of positive sample nodes is calculated based on the positive sample node embedding vectors. A similarity matrix is ​​constructed based on the similarity, and the top k highly similar neighbor nodes are selected for each node to construct new connection edges. The enhanced adjacency matrix in the current iteration is updated and obtained with the new connection edges. Cluster analysis is performed on the node embeddings containing cluster features in the attribute graph G based on the K-means algorithm to obtain the clustering results of the academic paper citation network or collaborative network objects to be clustered.

2. The academic paper clustering method based on cluster feature learning according to claim 1, characterized in that, Based on contrastive learning theory, positive and negative sample groups are constructed, including: The dropout operation is used to randomly mask the node title features in the node attribute matrix. The randomly masked node attribute matrix is ​​used as the current node attribute matrix. The enhanced adjacency matrix obtained in the previous iteration is used as the current adjacency matrix. An enhanced view is constructed based on the current node attribute matrix and the current adjacency matrix. The enhanced view is used as the current positive sample. The structure of the current node attribute matrix is ​​destroyed by randomly changing the order of nodes in the node attribute matrix to generate a destroyed map as the current negative sample.

3. The academic paper clustering method based on cluster feature learning according to claim 1, characterized in that, Based on the merging and aggregation of positive and negative sample node embedding vectors, and updating the encoder parameters by minimizing a predefined binary cross-entropy objective loss function, the following is included: First, the embedding vectors of positive and negative sample nodes are concatenated and merged. Then, the sample set is obtained by summing the feature vectors of each node title. Finally, the accuracy of the positive and negative sample node discrimination is measured using a preset binary cross-entropy target loss function.

4. The academic paper clustering method based on cluster feature learning according to claim 1 or 3, characterized in that, The pre-defined binary cross-entropy objective loss function is expressed as follows: in This represents the result of merging and aggregating the embedding vectors of node i, y i The label representing node i, when node i originates from a positive sample, y i =1, otherwise y i =0, where N is the number of nodes in the positive / negative samples.

5. The academic paper clustering method based on cluster feature learning according to claim 1, characterized in that, The similarity between pairs of positive sample nodes is calculated based on the embedding vectors of positive sample nodes, including: Based on the positive sample node embedding vectors, the similarity between pairwise node pairs is calculated using cosine distance.

6. The academic paper clustering method based on cluster feature learning according to claim 1, characterized in that, For each node, select the top k highly similar neighbor nodes to construct new connection edges. Use these new connection edges to update and obtain the enhanced adjacency matrix in the current iteration round, which includes: First, the KNN classification algorithm is used to retain k highly similar nodes as neighbors for each node in the similarity matrix, resulting in a relatively sparse adjacency matrix. Next, the relatively sparse adjacency matrix is ​​symmetrically normalized to maintain its symmetry. Then, a pre-defined update formula is used... To obtain the enhanced adjacency matrix in the current iteration round, where ε is the graph structure update weight hyperparameter, and A * Let A represent the adjacency matrix after symmetric normalization, and let A represent the adjacency matrix of the attribute graph G.

7. An academic paper clustering system based on cluster feature learning, characterized in that, The method described in claim 1 includes: a graph construction module, a graph encoding module, and a clustering analysis module, wherein... The graph construction module is used to represent the academic paper citation network or collaboration network objects to be clustered as an attribute graph G. Here, nodes represent academic papers, edges represent the citation relationships between academic papers in the citation network or the collaboration relationships between paper authors in the collaboration network, the edge relationships between all academic paper nodes form an adjacency matrix, and the node attribute matrix is ​​composed of the title feature vectors of each academic paper node. The graph encoding module is used to construct a graph encoder to encode all nodes of the attribute graph G into the feature space, and to iteratively train the graph encoder through a preset group discrimination loss function to learn and obtain the cluster features embedded in the nodes of the attribute graph G. The clustering analysis module is used to perform clustering analysis on the node embeddings containing cluster features in the attribute graph G based on the K-means algorithm, and to obtain the clustering results of the academic paper citation network or collaborative network objects to be clustered.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 5.

Images