A reviewer recommendation method based on graph neural network

Abstract

The present invention belongs to the field of knowledge management and application technology, and specifically relates to a reviewer recommendation method based on a graph neural network. The network of the present invention adds two networks to the output layer of the graph convolutional neural network, namely, learning networks for degree information and clustering coefficient information in the graph structure information. To accelerate learning convergence and make learning more in-depth, the present invention adds learnable parameters when performing the sum operation of the three loss functions. To enhance the graph neural network's understanding of network structure, the present invention adds network structure information of degree and clustering coefficient to the input layer of the entire NIGCN network.

Description

Technical Field

[0001] The present invention belongs to the field of knowledge management and application technology, and specifically relates to a reviewer recommendation method based on graph neural network. Background Art

[0002] For a newly submitted paper, the author's provided sub-field labels are generally used to determine its field and to identify relevant professionals for review. However, the sub-field labels provided by the author may not be accurate and comprehensive, and may only include labels that are beneficial to the paper's approval. Therefore, finding reviewers in the right field to provide a fair and comprehensive evaluation of the paper is a meaningful application problem. One feasible approach is to predict the field of the new paper based on the existing papers cited by the newly submitted paper and the fields of the existing papers, thereby providing a reference for how to find reviewers in the corresponding field. If papers are regarded as nodes and citation relationships as edges, then a paper citation graph is formed, and predicting the field of the paper becomes a graph node classification problem.

[0003] Graph node classification is a widely applicable task, generally categorized into supervised learning, semi-supervised learning, and unsupervised learning. Semi-supervised graph node classification is a very common task, and the primary focus of this paper. Given a graph and the labels of some of its nodes, semi-supervised classification involves learning from the graph structure and labeled nodes to predict the labels of other unlabeled nodes. This is a very broad task. For example, given a teacher-student relationship graph where some nodes are labeled as either teacher or student, semi-supervised classification can be used to predict whether other unlabeled nodes are teachers or students based on associations and labels.

[0004] Some traditional methods can be used to solve the semi-supervised graph node classification problem, such as:

[0005] Support Vector Machine (SVM): SVM is a classic machine learning algorithm that can be used for graph node classification. It performs classification by finding an optimal hyperplane in feature space. The structural features of graph nodes can be used as input feature vectors, and then the SVM is used for classification.

[0006] Random Forest: Random Forest is an ensemble learning method that can be used for graph node classification. It consists of multiple decision trees and performs classification through voting or averaging. The structural features of the graph nodes can be used as input feature vectors for classification using Random Forest.

[0007] K-Nearest Neighbors (KNN): The KNN algorithm is a nonparametric machine learning algorithm that can be used for graph node classification. It determines the node category by calculating the distance or similarity between nodes. The KNN algorithm can be used to classify nodes using the node feature vector as input.

[0008] While these traditional methods have achieved numerous impressive results, they also suffer from issues such as insufficient learning capabilities, limited flexibility, and poor generalization. Deep learning methods, which can overcome these shortcomings to a certain extent, are gaining increasing attention and widespread application. Graph convolutional neural networks are a prime example of this type of deep learning method.

[0009] Graph Convolutional Network (GCN) is a graph neural network that can directly act on graphs and effectively utilize the structural information of graphs. It effectively extracts the adjacent node information of input graph nodes with low computational complexity through efficient node aggregation. It has good interpretability in both spatial and spectral domains, and can quickly extract information on the graph through simple operations. It is an efficient graph neural network. According to general experience, such as Figure 1 As shown in the figure, the graph convolutional neural network containing two layers of graph convolutional layers has the highest prediction accuracy. Its structure is generally a series connection of graph convolutional layer 1, activation layer, dropout layer, and graph convolutional layer 2.

[0010] make Indicates the The graph feature matrix of the layer, Indicates the The graph feature matrix of the layer. Let Represents the degree matrix of the graph neural network input graph, that is, constructing a diagonal matrix , for any node , assign its degree value to , the resulting matrix is ​​the degree matrix of the graph. Let Denote the adjacency matrix of the graph. Let Denotes the identity matrix. Let , For any invertible matrix , . The matrix represents the learnable parameters of this GCN layer. Represents the activation function of the neural network in this layer.

[0011] The overall information transmission form of the currently commonly used graph convolutional neural network is

[0012]

[0013]

[0014] The dimensions can be calculated , is the dimension of the node feature vector, is a hyperparameter. It has nothing to do with the number of nodes. The advantage of this is that it can handle dynamic graphs where nodes change, as long as the characteristic dimension of the node remains unchanged, the parameter matrix The dimension remains unchanged.

[0015] but The lack of dependency on the number of nodes also creates a problem: the parameters don't adequately learn the graph structure, especially the structure associated with the graph nodes, which results in insufficient graph neural network learning. Furthermore, when aggregating node messages and multiplying them by the parameter matrix, it can be seen that the graph structure information (degree) in the message aggregation is all in the denominator, while the numerator lacks graph structure information. Summary of the Invention

[0016] To address these issues, the present invention proposes a graph convolutional neural network (NIGCN) based on embedded network structure information. Two networks are added to the output layer of the NIGCN: networks that learn the degree and clustering coefficient information contained in the graph structure information. To accelerate learning convergence and deepen learning, the present invention adds learnable parameters to the sum of the three loss functions. To enhance the NIGCN's understanding of network structure, the present invention adds network structure information, such as degree and clustering coefficient, to the input layer of the entire NIGCN network.

[0017] The technical method of the present invention is:

[0018] A reviewer recommendation method based on graph neural network, including the following steps:

[0019] S1. Construct training data based on the cora dataset, and use the feature matrix, clustering coefficient and degree information of the graph constructed by the cora dataset as training data;

[0020] S2. Construct a reviewer recommendation model based on a graph neural network, including the first graph convolution layer, the first activation layer, the second graph convolution layer, the Softmax layer, the first linear layer, the second activation layer, the third linear layer, the fourth linear layer, the fifth activation layer, and the sixth linear layer; the feature matrix, clustering coefficient, and degree are jointly input into the first graph convolution layer and the first activation layer, which serve as a common hidden layer, and then the data output by the first activation layer are input into three parallel branches, namely, the common hidden layer output data is input into the first branch composed of the second graph convolution layer and the Softmax layer, and the first branch outputs the learning result of the classification label value; the common hidden layer output data and the degree data are jointly input into the second branch composed of the first linear layer, the second activation layer, and the third linear layer, and the second branch outputs the learning result of the degree information; the common hidden layer output data and the clustering coefficient data are jointly input into the third branch composed of the fourth linear layer, the fifth activation layer, and the sixth linear layer, and the third branch outputs the learning result of the clustering coefficient; the second and third branches are auxiliary training and do not participate in the prediction of the results after training. Only the results output by the first branch during recommendation are used for prediction;

[0021] S3, uses the training data constructed in S1 to train the recommendation model constructed in S2, and the loss function used is:

[0022]

[0023] As described in S2, the first branch outputs the classification label value, and the actual label value is subjected to the normalized cross entropy loss function to obtain loss1. The second branch outputs the degree information, and the actual degree information is subjected to the mean square error loss function to obtain loss2. The third branch outputs the clustering coefficient, and the actual clustering coefficient is subjected to the mean square error loss function to obtain loss3. There are three learnable parameters;

[0024] After training, a trained recommendation model is obtained;

[0025] S4: Use the trained recommendation model to make recommendations. Specifically, the graph feature information generated by the cora dataset (i.e., whether a newly submitted paper contains a certain word), the paper citation graph, and the degree and clustering coefficient of the paper node are input into the trained recommendation model. Only the output of the learning result of the first branch described in S2 is used as the recommendation result.

[0026] The beneficial effect proposed by the present invention is that: while retaining the advantage of the graph convolutional neural network GCN in dealing with dynamic graphs, the present invention enables the parameters to learn more network structure information, thereby improving the neural network's ability to understand the graph, thereby improving the prediction accuracy of the field of newly submitted papers. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 shows the operation process of a two-layer graph convolutional neural network;

[0028] Figure 2 shows the basic part of the NIGCN network, a two-layer graph convolutional neural network;

[0029] Figure 3 is a diagram of the NIGCN network structure;

[0030] Figure 4 shows the learning part of the degree;

[0031] Figure 5 shows the comparison of the maximum prediction accuracy of the NIGCN network and the basic two-layer GCN network in 20 experiments;

[0032] Figure 6 compares the average prediction accuracy of the NIGCN network and the basic two-layer GCN network in 20 groups of experiments. DETAILED DESCRIPTION

[0033] The present invention is further described in detail below with reference to the accompanying drawings and simulation examples.

[0034] The network NIGCN of the present invention is as follows Figure 2 The two-layer graph convolutional neural network (GCN) shown is used as the basis, and two networks are added to the output layer of the graph convolutional neural network, which are learning networks for the degree information and clustering coefficient information in the graph structure information. In order to strengthen the learning of network structure information, the input of the network part added by the present invention is a combination of network structure information and the feature vector learned by the hidden layer, that is, the input of the two output layer networks is the combination of degree and hidden layer output and the combination of clustering coefficient and hidden layer output, respectively. Unlike classification tasks, these two tasks are regression learning, so the output layer network adopts a "linear layer" + a "Relu activation layer" + "a linear layer" method. This simple network form can achieve an improvement in network learning effect and takes up less computing resources. The final output of the two networks added to the output layer is the learned network science information.

[0035] like Figure 3 As shown, the NIGCN network of the present invention will simultaneously output the learning of node labels, degrees and clustering coefficients, and calculate the loss functions of the three respectively. Convert the three loss functions to the same order of magnitude and add them together, then backpropagate this information.

[0036] In order to accelerate the convergence of learning and make learning more in-depth, the present invention adds learnable parameters when performing the sum operation on the three loss functions. After testing various parameter addition methods, it was found that multiplying the square of the loss function before the three loss functions and adding the square of the loss function after the formula has the highest accuracy in predicting the graph node labels. The model of the present invention adopts this form. Assume that the three learnable parameters are , then the final loss function The form is:

[0037]

[0038] In order to enhance the understanding of the network structure, the present invention adds the network structure information of degree and clustering coefficient to the input layer of the entire NIGCN network.

[0039] like Figure 4 The figure shows the network structure of the learning part of participation, and the symbolic representation of each parameter is also shown in the figure. use express, Indicates that the degree is expressed in degrees, and the input of convolutional layer 1 is expressed in Indicates that the output after the first Relu activation layer is Indicates that the linear layer 1 is represented by Indicates that the output of linear layer 1 is expressed as Indicates that the output after the second Relu activation layer is Indicates that the linear layer 2 is used Indicates that the learning express.

[0040] Represents the concatenation of three vectors. Represents the output of the network's learning of degree, and is distinguished from the true degree information The least squares loss function used in this regression learning task is represented by e, and the formula is:

[0041]

[0042] make is a 0-1 vector, The values ​​are:

[0043]

[0044]

[0045] Similarly, is a 0-1 vector,

[0046]

[0047] is the Hadamard Product, defined as the matrix are two matrices of the same order, if , then it is called the matrix is the Hadamard product of A and B.

[0048] Will Remove the last column, that is The coefficients of , we get the matrix

[0049] As mentioned in the GCN section above,

[0050] According to the back propagation principle of the neural network, the information transmission path in the figure above is deduced, and the result is:

[0051]

[0052] According to the forward propagation,

[0053]

[0054] According to the gradient descent formula:

[0055]

[0056] Many parameters have unclear meanings.

[0057] from and It can be seen from the derivation results that the degree information is enhanced to the graph convolutional neural network parameters after parameter operation. Furthermore, during the information transfer process, degree information is introduced at multiple levels of operations at multiple locations in the network, making the learning process more focused on degree information and the degree information learned for each parameter more comprehensive. The same principle applies to learning the clustering coefficient. As described in the background section on graph convolutional neural networks, what graph convolutional neural networks lack is network structure information at the molecular level in the parameter part. Both degree and clustering coefficient provide network structure information that is closely related to graph nodes and can therefore effectively improve network performance.

[0058] Through the back-propagation process, the NIGCN network of the present invention not only maintains the advantage of the graph convolutional neural network (GCNN) that the parameter dimensions are independent of the nodes and can cope with dynamic graphs, but also strengthens the parameter matrix's learning of network structure information, improves the GCNN's understanding of the graph, and thus improves performance. At the same time, the increase in computational complexity is minimal, making it possible to cope with large-scale graphs.

[0059] Simulation example:

[0060] The CORA dataset, commonly used for network node classification, contains 2,708 scientific publications, 5,429 edges, and seven categories. Each publication in the dataset is described by a 0 / 1 word vector, indicating the absence / presence of the corresponding word in the dictionary. The dictionary consists of 1,433 unique words. This means that each publication is composed of 1,433 features, each represented by a 0 / 1 value. In other words, the CORA dataset has a total of 2,708 nodes, and the input is a 2,708-by-1, 0 / 1 feature matrix, and the labels are a 2,708-by-7, 0 / 1 matrix.

[0061] The average accuracy of the basic graph convolutional neural network (GCN) on this dataset is approximately 80.5%, and the highest accuracy is approximately 81.5%. The current best accuracy of the present invention is 83.6%.

[0062] After many simulation experiments, such as Figure 5 and Figure 6 As shown, the NIGCN network of the present invention achieves a maximum prediction accuracy of 83.1% and an average accuracy of 82%. This compares to the maximum accuracy of approximately 81.5% and the average accuracy of 80.5% achieved by the underlying graph convolutional neural network (GCN). This demonstrates that the modifications made in the present invention significantly improve network performance. Furthermore, the output of the present invention is more stable, achieving a more robust training effect.

[0063] The experiment designed by the present invention divides the cora dataset into a training set and a test set according to the default method. Because predictions on the test set are uncertain and fluctuate, the maximum value, mean value, and variance are all meaningful features. The experiment designed by the present invention runs the NIGCN network of the present invention and a basic graph convolutional neural network (GCN) simultaneously in the same program, and records the data. This ensures a consistent experimental environment, making the comparison of results more effective. The experimental method employed by the present invention initializes, trains, and tests the NIGCN network of the present invention to obtain the accuracy of graph node classification predictions. This process is repeated 20 times as a group, and the average and maximum prediction accuracy of each group are output. The basic graph convolutional neural network is also processed as above and the corresponding results are output. Finally, the above process is repeated 20 times, and the output data is compared with a line chart for analysis. In this experiment, the number of training epochs for the NIGCN network of the present invention is 150, and the number of training epochs for the basic network graph convolutional neural network is 80, which is required for convergence. The reason for choosing 150 training epochs for the NIGCN network of the present invention is discussed below in the section on overfitting. Table 1 below shows the average accuracy and maximum accuracy of the NIGCN network of the present invention and the reference basic network graph convolutional neural network in 20 groups of experiments.

[0064] Table 1

[0065] Group number NIGCN network maximum Basic network maximum value NIGCN network average Basic network average 1 0.828 0.812 0.8215 0.8055 2 0.830 0.814 0.8206 0.8053 3 0.828 0.811 0.8196 0.8035 4 0.825 0.817 0.8202 0.8052 5 0.828 0.812 0.8200 0.8041 6 0.826 0.814 0.8196 0.8045 7 0.825 0.818 0.8191 0.8064 8 0.826 0.814 0.8190 0.8055 9 0.827 0.809 0.8208 0.8030 10 0.825 0.809 0.8187 0.8046 11 0.825 0.812 0.8195 0.8046 12 0.824 0.814 0.8182 0.8041 13 0.828 0.812 0.8211 0.8058 14 0.827 0.817 0.8200 0.8073 15 0.827 0.815 0.8197 0.8060 16 0.827 0.818 0.8189 0.8064 17 0.827 0.813 0.8189 0.8048 18 0.825 0.813 0.8206 0.8060 19 0.830 0.815 0.8199 0.8057 20 0.828 0.816 0.8209 0.8055

[0066] For the four columns of data in the table above, namely the maximum value of the NIGCN network, the maximum value of the basic network, the average value of the NIGCN network, and the average value of the basic network, we calculated the average values ​​and variances of 20 groups of experiments.

[0067] Average value of the maximum value of NIGCN network: ,

[0068] Average value of the base network maximum: ,

[0069] Variance of the maximum value of the NIGCN network: ,

[0070] Variance of the maximum value of the base network: ,

[0071] The average of the NIGCN network averages (also the average of 400 training runs): ,

[0072] The average of the base network averages (also the average of 400 training runs): ,

[0073] Variance of the NIGCN network average:

[0074] Variance of the base network mean: ,

[0075] The data in the table and the calculation of data features reveal the following conclusions. In this experiment, the NIGCN network of the present invention achieved a maximum prediction accuracy of 83.0%. In comparison, the basic graph neural network achieved a maximum accuracy of 81.8%. This experiment demonstrates that the NIGCN network of the present invention improved the maximum accuracy by 1.2%. The average of the maximum values ​​across 20 experimental groups was 82.68% for the NIGCN network of the present invention and 81.38% for the basic network, a 1.3% improvement in the average of the maximum values. The average of the 20 average values ​​(i.e., the average of 400 training runs) showed that the NIGCN network of the present invention achieved an average of 82%, while the basic network achieved an average of 80.52%, an improvement of nearly 1.5%. The variance of the maximum and average values ​​of the NIGCN network of the present invention and the basic network was smaller, indicating that the trained performance of the network of the present invention is more stable.

[0076] In summary, the NIGCN network implemented in this invention significantly outperforms the baseline network in terms of prediction accuracy for the prediction set. Not only does it significantly improve both the peak accuracy and the average accuracy, but the trained network also exhibits more stable prediction accuracy. The NIGCN network of this invention offers significant advantages.

[0077] By selecting some other models, including graph convolutional neural network GCN, graph attention neural network GAT, simple graph convolutional neural network SGC, MixHop network, GraphSAGE network, fast graph convolutional neural network FastGCN, The highest accuracy of these models on the cora dataset was compared with the NIGCN model of the present invention. The results are shown in Table 2.

[0078] Table 2

[0079] Model Name Prediction accuracy NIGCN 83.1% GCN 81.5% GAT 83.0% SGC 81.0% MixHop 81.9% GraphSAGE 78.9% FastGCN 81.4% <![CDATA[G 3 NN]]> 82.5% DropEdge 82.8%

[0080] This demonstrates that the NIGCN model proposed in this paper surpasses the performance of many mainstream models while retaining the advantages of graph convolutional neural networks (GCNNs)—fast training and ability to handle dynamic graphs. Generally speaking, models that outperform the model proposed in this paper require significantly higher computing resources. For example, in our environment, the GRAND model requires approximately half an hour to train, while the NIGCN model takes less than half a minute. This demonstrates the significant application value of the NIGCN model proposed in this paper in applications where computing power is limited.

Claims

1. A reviewer recommendation method based on graph neural network, characterized in that: The following steps are involved: S1. Construct training data based on the cora dataset, and use the feature matrix, clustering coefficient and degree information of the graph constructed by the cora dataset as training data; S2. Construct a reviewer recommendation model based on a graph neural network, including a first graph convolution layer, a first activation layer, a second graph convolution layer, a Softmax layer, a first linear layer, a second activation layer, a third linear layer, a fourth linear layer, a fifth activation layer, and a sixth linear layer; the feature matrix, clustering coefficient, and degree are jointly input into the first graph convolution layer and the first activation layer as a shared hidden layer, and then the data output from the first activation layer is input into three parallel branches, namely, the shared hidden layer output data is input into the first branch composed of the second graph convolution layer and the Softmax layer, and the first branch outputs the learning result of the classification label value; the shared hidden layer output data and the degree data are jointly input into the second branch composed of the first linear layer, the second activation layer, and the third linear layer, and the second branch outputs the learning result of the degree information; The shared hidden layer output data and the clustering coefficient data are inputted into a third branch consisting of a fourth linear layer, a fifth activation layer, and a sixth linear layer, and the third branch outputs the learning result of the clustering coefficient; S3, uses the training data constructed in S1 to train the recommendation model constructed in S2, and the loss function used is: ， Among them, the first branch outputs the learning result of the classification label value and the actual label value through the normalized cross entropy loss function to obtain loss1, the second branch outputs the learning result of the degree information and the actual degree information through the mean square error loss function to obtain loss2, and the third branch outputs the learning result of the clustering coefficient and the actual clustering coefficient through the mean square error loss function to obtain loss3. There are three learnable parameters; After training, a trained recommendation model is obtained; S4. Use the trained recommendation model to make recommendations. Specifically, the graph information of the newly submitted paper and its cited papers is input into the trained recommendation model to obtain the recommendation results.

Images