A literature classification method and system based on a twin graph neural network and a cluster structure

Abstract

The application discloses a literature classification method and system based on a twin graph neural network and a cluster structure, and the method comprises the following steps: detecting a cluster structure in a citation network by using a clustering algorithm to obtain cluster structure information; adaptively modulating the weight of the connection between clusters; propagating the features of the citation network and the original citation network after the weight modulation through a twin graph neural network structure to obtain neighbor node information and obtain two different feature representations of the literature nodes; inputting the features with the cluster information after the feature propagation and the original features into a feature fusion module for feature fusion; and inputting the fused features into a linear multi-classifier to obtain a final classification result. The application effectively improves the classification accuracy by retaining the cluster structure in the citation network.

Description

Technical Field

[0001] This invention belongs to the field of document classification and relates to a document classification method and system based on twin graph neural networks and cluster structures. Background Technology

[0002] Many real-world applications generate data from non-Euclidean spaces, and traditional deep learning methods struggle to perform satisfactorily when processing such data. By defining convolution operators between graphs and signals, Graph Convolutional Networks (GCNs) extend Convolutional Neural Networks (CNNs) to graph-structured inputs that contain attributes.

[0003] However, GCN suffers from long training times, a large number of model parameters, and the problem that the model's performance becomes overly smoothed as the depth increases. Therefore, it only considers each node's neighbors within a few hops, which is insufficient to capture the global cluster structure of large graphs. This limits its ability to aggregate features of high-order neighbor nodes, resulting in a decrease in classification accuracy. Summary of the Invention

[0004] Objective: To overcome the shortcomings of existing technologies, this invention provides a document classification method and system based on Siamese graph neural networks and clustering algorithms. The clustering algorithm and Siamese neural network preserve the cluster structure in the citation network, and a linear multi-classifier is used for document classification.

[0005] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0006] Firstly, a document classification method based on Siamese graph neural networks and cluster structures is provided, including: using a clustering algorithm to detect cluster structures in the graph and obtain cluster structure information; adaptively modulating the weights of connections between clusters; propagating the weighted citation network and the original citation network through a Siamese graph neural network structure to obtain neighbor node information and obtain a node feature matrix; fusing the features with cluster structures after feature propagation with the original features into a feature fusion module; and inputting the fused features into a linear classifier to obtain the final classification result.

[0007] Furthermore, a similarity matrix of document nodes in the citation network is constructed using a Gaussian kernel function, and a spectral clustering algorithm is used to assign a category label to each document node. Document nodes with the same label belong to the same cluster, thus obtaining cluster information.

[0008] Furthermore, the average value of the document node features within each cluster is taken as the center of the cluster. The Euclidean distance d of each cluster center is calculated. w = 1 / d is used as the weight of the connection between clusters and normalized to ensure that the weight of the connection between clusters is between 0 and 1, while the weight of the connection within the cluster remains unchanged and is 1.

[0009] Furthermore, the weighted citation network and the original citation network are input into the simplified graph neural network SGC. SGC significantly reduces the computational load by removing the nonlinear activation function of the graph neural network GCN. After multi-layer feature propagation, node features H1 with cluster information and the original features H2 are obtained.

[0010] Furthermore, the features with cluster information after feature propagation are fused with the original features input feature fusion module, including averaging H1 and H2 to obtain the final node output feature H0.

[0011] Furthermore, the fused features H0 are input into a softmax multi-classifier, and the final classification result Y = (y1, y2, ..., y0) is obtained according to the formula Y = softmax(H0Θ). n ), where n is the number of documents, y i ,i=1,2…n are the category labels of the documents, and Θ is the weight matrix.

[0012] Secondly, a document classification system based on Siamese graph neural networks and cluster structures is provided, including a cluster detection module for detecting cluster structures in the citation network; an adaptive weight modulation module for weight modulation of inter-cluster connections in the citation network to preserve the cluster structure; a feature fusion module for propagating the feature matrices of document nodes using both the weight-modulated citation network and the original citation network, fusing neighbor node information; and also for fusing cluster information into the original information representation; and a document classification module for classifying documents using the processed feature matrices of document nodes.

[0013] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0014] This invention utilizes a clustering algorithm to detect the cluster structure in a citation network and obtain cluster structure information; it adaptively modulates the weights of the connections between clusters; it then propagates the weight-modulated citation network and the original citation network through a Siamese graph neural network structure to obtain neighbor node information and obtain a node feature matrix; it fuses the features with cluster structure obtained after feature propagation with the original features into a feature fusion module; and finally, it inputs the fused features into a linear classifier to obtain the final classification result, effectively improving the classification accuracy. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the main process of a document classification method based on twin graph neural network and cluster structure provided by an embodiment of the present invention;

[0016] Figure 2 This is a schematic diagram of adaptive weight modulation in an embodiment of the present invention. Detailed Implementation

[0017] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0018] Example 1:

[0019] like Figure 1 and Figure 2 As shown, a document classification method based on Siamese graph neural network and cluster structure includes: using a clustering algorithm to detect the cluster structure in the citation network and obtain cluster structure information; adaptively modulating the weights of the connections between clusters; performing feature propagation on the weight-modulated citation network and the original citation network through a Siamese graph neural network structure to obtain neighbor node information and obtain a node feature matrix; fusing the features with cluster structure after feature propagation with the original features into a feature fusion module; and inputting the fused features into a linear classifier to obtain the final classification result.

[0020] like Figure 1 As shown, for the citation network, we first use spectral clustering to detect its cluster structure, and then... Figure 2 The adaptive weight modulation method shown modulates the inter-cluster connections. Then, the feature matrices of the document nodes are input into both the upper and lower branches of the Siamese graph neural network. In the upper branch, we perform k-fold feature propagation using the weight-modulated citation network; in the lower branch, we perform k-fold feature propagation using the original citation network. Next, the node feature matrices from the propagation of the upper and lower branches are fused, and finally input into a multi-classifier for document classification. Specifically, the following steps are included:

[0021] S1. Use clustering algorithms to detect cluster structures in citation networks and obtain cluster structure information; adaptively modulate the weights of connections between clusters.

[0022] S11. Cluster detection is performed on the input citation network using a spectral clustering algorithm, where the similarity matrix of document nodes is calculated using a Gaussian kernel function. Construct, where v i and v j w represents the feature vector of a document node. ij Represents node v i and vj The similarity between them.

[0023] S12, such as Figure 2 As shown, the center point of each cluster is calculated using the cluster structure detected in S11, and then the Euclidean distance d between the cluster center points is calculated. i :

[0024] d ij =||v i -v j ||.

[0025] Since the greater the distance between clusters, the lower the amount of information in the inter-cluster connections, the weights of the inter-cluster connections are modulated to w. ij =1 / d ij Since the weights should be within the interval [0,1], we normalized the weights. The normalized weights are w. ij =w ij / max(W), where W is the weight matrix of the cluster.

[0026] S2. Propagate the feature matrix of the document node through a twin graph neural network structure to obtain neighbor node information; fuse the features with cluster structure after feature propagation with the original features input feature fusion module.

[0027] S22. Input the feature matrix of the document node into the Siamese graph neural network. In the upper branch, we use the weighted citation network to perform k feature propagation to obtain the feature matrix H1. In the lower branch, we use the original citation network to perform k feature propagation to obtain the feature matrix H2. Then, we perform feature fusion on the node feature matrices after propagation in the upper and lower branches. The feature fusion operation is to average the features of the two feature matrices H0 = (H1 + H2) / 2, and then integrate the cluster structure into the output node features.

[0028] S3. Input the fused features into a linear classifier to obtain the final classification result.

[0029] S31. Input the fused features H0 into the softmax multi-classifier, and obtain the final classification result Y = (y1, y2, ..., y0) according to the formula Y = softmax(H0Θ). n ), where n is the number of documents, y i ,i=1,2…n are the category labels of the documents, and Θ is the weight matrix.

[0030] This invention proposes a clustering algorithm to detect the cluster structure in a citation network and adaptively selects the weights of connections between clusters. To explicitly consider the cluster information in the citation network, a Siamese graph neural network framework is proposed to fuse cluster structure information into the original information representation, which helps improve classification accuracy. Experiments on the citation datasets Cora, Citeseer, and PubMed demonstrate that the final document classification accuracy can reach 81.8%, 71.3%, and 79.9%, respectively.

[0031] Example 2:

[0032] Based on the document classification method based on Siamese graph neural network and cluster structure described in Embodiment 1, this embodiment provides a document classification system based on Siamese graph neural network and cluster structure, including:

[0033] The cluster detection module is used to detect cluster structures in the citation network.

[0034] The adaptive weight modulation module is used to modulate the weights of the inter-cluster connections in the citation network while preserving the cluster structure of the citation network.

[0035] The feature fusion module is used to propagate the feature matrices of document nodes using both the weighted citation network and the original citation network, and to fuse the information of neighboring nodes.

[0036] It is also used to combine the features that retain cluster information and the original features by averaging them, and then integrate the cluster information into the original information representation.

[0037] The document classification module is used to classify documents by inputting the feature matrix of the processed document nodes into a softmax multi-classifier.

[0038] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A document classification method based on twin graph neural networks and cluster structure, characterized in that, The method comprises: detecting cluster structure in the citation network by using a clustering algorithm to obtain cluster structure information; performing adaptive modulation on the weight of the connection between clusters to obtain a citation network after weight modulation; performing feature propagation on the citation network after weight modulation and the original citation network through a twin graph neural network structure to obtain neighbor node information and obtain two different feature representations of the literature nodes; inputting the features after feature propagation and having cluster information into a feature fusion module for feature fusion; inputting the fused features into a linear multi-classifier to obtain a final classification result; detecting cluster structure in the citation network by using a clustering algorithm to obtain cluster structure information, comprising: constructing a similarity matrix of the literature nodes in the citation network by using a Gaussian kernel function, and assigning a category label to each literature node by using a spectral clustering algorithm, the literature nodes with the same label belong to the same cluster, and obtaining cluster information; performing adaptive modulation on the weight of the connection between clusters, comprising: taking the average value of the features of the literature nodes in each cluster as the center of the cluster, calculating the Euclidean distance d of each cluster center, taking w=1 / d as the weight of the connection between clusters and performing normalization processing, formula w=w / max(w), to ensure that the weight of the connection between clusters is between 0 and 1.

2. The document classification method based on twin graph neural networks and cluster structure according to claim 1, characterized in that, performing feature propagation on the citation network after weight modulation and the original citation network through a twin graph neural network structure to obtain neighbor node information and obtain two different feature representations of the literature nodes, comprising: The citation network after weight modulation and the original citation network are respectively input into a simplified graph neural network SGC, and node features with cluster structure are obtained after multi-layer feature propagation and the original features .

3. The document classification method based on twin graph neural networks and cluster structure according to claim 1, characterized in that, The features with cluster structure after feature propagation are input into a feature fusion module together with original features for feature fusion, including: and taking average as the final node output feature .

4. The document classification method based on twin graph neural networks and cluster structure according to claim 1, characterized in that, inputting the fused features into a linear multi-classifier to obtain a final classification result, comprising: The fused features are input into a softmax multi-classifier according to the formula to obtain the final classification result Y, where is a weight matrix.

Images