A seed set expansion method and system based on graph neural network

Abstract

The present invention discloses a seed set expansion method and system based on a graph neural network, which specifically includes the following steps: Step 1, for given graph data, obtain graph structure information and node attributes, given pseudo labels, seed sets and the number of nodes to be expanded K, train two graph neural networks through the EM algorithm, and calculate the attention coefficient of the pseudo labels and the attention coefficient of the node features; Step 2, calculate the probability matrix A according to the attention coefficient of the pseudo labels and the attention coefficient of the node features; Step 3, expand the number of nodes K as needed, use the probability coefficient in the probability matrix A to guide PageRank to walk, and obtain the nodes expanded. The present invention combines label and node attribute information and is applied in the seed set expansion process. According to experiments, compared with the traditional seed set expansion algorithm, the expanded node set has more nodes belonging to the community where the seed set is located.

Description

Technical Field

[0001] The present invention belongs to the technical field of big data management and analysis, and specifically relates to a seed set expansion method and system based on a graph neural network, which are used to expand the seed set of a graph according to known nodes. Background Art

[0002] With the rise of the Internet, more and more networks have attracted widespread attention. Networks have become increasingly important in modeling various complex systems. Generally speaking, networks are composed of vertices (individuals) and edges (relationships between individuals), and networks can be abstracted into graphs. In application scenarios such as product recommendation and criminal gang identification, people are interested in the nodes of a community (a set of similar nodes) in the network. These nodes have similarities in some cases, and usually people only know a small number of nodes in the community. Given a small number of nodes in a graph as a seed set, the seed set is expanded into a larger set to approach the full range of the community. The expanded nodes must meet the conditions similar to the seed set as much as possible. There are many methods that can be used to solve the seed set expansion problem, which can be divided into four categories. The first category: There are three methods for counting neighbor nodes. Neighbors and DN-Neighbors refer to the number of neighbor nodes under a known community label, Outwardness calculates the difference between the number of degrees in the community and the number of degrees outside the community, and Binomial Prob calculates the binomial probability of node neighbors in the community. The second category: There are three algorithms that use the greedy idea. Modularity calculates a value and selects the node with the highest value as the expansion node each time. It is an important indicator for measuring community division and can also be used in seed set expansion. Set-Modularity deletes nodes that increase its calculated value and adds nodes that reduce its calculated value. Conductance calculates the ratio of edges within the community to edges between communities, which is an important criterion for measuring community quality. The third category: The method using random walks includes PageRank. It is stated in existing literature that PageRank can achieve the best expansion effect. The fourth category: The Augmented label propagation (ALP) algorithm relies on the network structure and uses the ideas of dynamic propagation and label propagation to learn node expansion.

[0003] Traditional seed set expansion algorithms mainly use label propagation and graph topology for expansion. Graph topology knows the connection information between nodes, including out-degree, in-degree and other information. Important nodes are mainly calculated based on information such as out-degree, in-degree and number of edges, or the wandering method is determined by the number of edges, and the nodes most similar to the seed set are selected as the expanded nodes through continuous iteration. However, it is obvious that the network in the real world not only has structural information, but also carries a variety of attribute information. It is not difficult to understand that this information can play a positive role in expansion. For example, in the citation network, the node attributes represent the important words in the article abstract. The same key words may appear in the literature in the same research field. These key words can play a key role in finding similar literature. Therefore, for most graph data, the use of node attributes can be a good help for seed set expansion, but in the traditional seed set expansion method, there is no way to combine node attributes to expand the seed set, which makes the number of nodes in the expanded node seed set small. Summary of the invention

[0004] In view of the problem that the above-mentioned existing seed set expansion algorithm does not use node attributes to expand the seed set, resulting in a small number of nodes belonging to the community where the seed set is located, the purpose of the present invention is to provide a seed set expansion method and system based on graph neural network. The present invention uses graph neural network (GNN) to assist in training the structural information of the graph, node attribute information and pseudo-labels, and uses this information from the perspective of graph neural network to learn the attention coefficients about node attributes and the attention coefficients about labels. After the attention coefficients are normalized, the processed results are used as a guide to provide a biased walking mode for PageRank walking, so that the node does not randomly walk to a neighbor node during the walking process, but preferentially walks to the neighbor nodes with closer relationships, thereby achieving a better expansion effect.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] On the one hand, the present invention provides a seed set expansion method based on graph neural network, which specifically includes the following steps:

[0007] Step 1: For the given graph data, obtain the graph structure information and node attributes, and given the pseudo-label, seed set, and the number of nodes to be expanded K, train two graph neural networks through the EM algorithm to calculate the attention coefficient of the pseudo-label and the attention coefficient of the node feature;

[0008] Step 2: Calculate the probability matrix A based on the attention coefficient of the pseudo-label and the attention coefficient of the node feature;

[0009] Step 3: Expand the number of nodes K as needed, use the probability coefficients in the probability matrix A to guide PageRank to wander, and obtain the nodes that are expanded to.

[0010] Furthermore, the step 1 specifically includes the following sub-steps:

[0011] Step 11, obtaining graph structure information, node attributes, pseudo labels, seed sets, and the number of nodes K that need to be expanded; the seed set is obtained by taking a certain proportion of nodes of the same category in the data set; wherein the graph structure information includes graph node IDs and edge connection information; the number of nodes K that need to be expanded is given; the node pseudo labels are given, and the initial pseudo label matrix is ​​an n×n unit matrix;

[0012] Step 12, the initial pseudo label matrix and the adjacency matrix are iterated cyclically to obtain a new pseudo label matrix PL;

[0013] Step 13, pre-training the E step: input the graph structure information, node attributes, and pseudo-label matrix PL to pre-train the GNN model; obtain the pre-trained GNN model and the predicted labels of the unknown label nodes;

[0014] Step 14, perform the first M-step training: input the graph structure information, the pseudo-label matrix PL, and use the predicted labels of the unknown label nodes as the true labels of the loss function, train the GNN model to obtain the training model and predicted labels after the first M-step training;

[0015] Perform the first E-step training: Use the predicted label after the first M-step training as the true label of the loss function of the first E-step training, input the graph structure information and node features, perform E-step training on the pre-trained GNN model, and obtain the predicted label after the first E-step training;

[0016] Perform the second M-step training: Use the predicted label after the first E-step training as the true label of the loss function of the second M-step training, and input the graph structure information and pseudo label matrix PL. Perform the second M-step training on the model after the first M-step training to obtain the predicted label of the second M-step training and the attention coefficient of the pseudo label. The collection of pseudo labels here has an attention coefficient Obtained from the second layer of the GAT model in the GNN model;

[0017] Perform the second E-step training: Use the predicted label after the second M-step training as the true label of the loss function of the second E-step training, input the graph structure information and node features, perform the second E-step training on the model after the first E-step training, and obtain the attention coefficient of the node features of the second E-step training A collection of node features, where the attention coefficient is It can be obtained in the second layer GAT model in the GNN model.

[0018] Furthermore, the step 12 specifically includes the following operations:

[0019] Step 1: Use the initial pseudo-label matrix as the preds matrix;

[0020] Step 2: Use the result obtained by multiplying the current preds matrix with the adjacency matrix to replace the current preds matrix; then, use the result obtained by adding the current preds matrix to the pseudo-label matrix to replace the current preds matrix;

[0021] Step 3: After setting the values ​​of the elements in the current preds matrix output in the second step that are greater than 1 to 1, the resulting matrix is ​​used as the current preds matrix;

[0022] Step 4: Repeat steps 2 and 3 for a total of 2 times (i.e., steps 2 and 3 in step 12 are performed 3 times in total), and assign the preds matrix outputted last time to the pseudo-label matrix to obtain a new pseudo-label matrix PL.

[0023] Furthermore, in the GNN model in step 13 and step 14, the first layer is a GAT model, the second layer is also a GAT model, and the third layer is a fully connected layer.

[0024] Furthermore, the step 2 specifically includes the following sub-steps:

[0025] Step 21, use formula (11) to calculate the mean attention coefficient E of node i and neighbor node j regarding node features and pseudo labels: ij :

[0026]

[0027] Step 22: Use formula (12) to calculate the mean sum of attention coefficients SE of node i and all neighboring nodes regarding features and pseudo labels: i :

[0028]

[0029] Step 23, use formula (13) to calculate the probability coefficient of node i wandering to neighbor node j; and combine the probability coefficients of all nodes and their neighbor nodes into a probability matrix according to the adjacency matrix of the graph

[0030]

[0031] Furthermore, the step 3 specifically includes the following sub-steps:

[0032] Step 31, the probability matrix A obtained in step 2 is used as a probability transfer matrix, and a personalized PageRank algorithm is used to process the probability matrix A to obtain the Pr value of each node;

[0033] Step 32, sort each node Pr outside the obtained seed set in descending order, and extract K / T nodes with the highest Pr value; K is the number of nodes to be expanded, which is a positive integer, and T is the number of cycles of the expansion process;

[0034] Step 33, adding the obtained nodes to the seed set to obtain a new seed set;

[0035] Step 34, determine whether the number of loops reaches T times, if yes, execute step 35, otherwise return to step 31; after T loops, a total of K+|S| nodes are obtained;

[0036] Step 35: remove the nodes in the initial seed set from the obtained nodes, a total of |S| nodes, and obtain the node set N consisting of K nodes expanded to K , the expansion ends.

[0037] On the other hand, the present invention provides a seed set expansion system based on graph neural network, which specifically includes the following modules:

[0038] The attention coefficient calculation module is used to implement the following functions: for given graph data, obtain graph structure information and node attributes, given pseudo labels, seed sets, and the number of nodes to be expanded K, train two graph neural networks through the EM algorithm, and calculate the attention coefficients of pseudo labels and node features;

[0039] A probability matrix calculation module is used to calculate the probability matrix A according to the attention coefficient of the pseudo label and the attention coefficient of the node feature;

[0040] The node expansion module is used to expand the number of nodes K as needed, use the probability coefficients in the probability matrix A to guide PageRank to wander, and obtain the expanded nodes.

[0041] Furthermore, the attention coefficient calculation module is specifically used to implement the following process:

[0042] Step 11, obtaining graph structure information, node attributes, pseudo labels, seed sets, and the number of nodes K that need to be expanded; the seed set is obtained by taking a certain proportion of nodes of the same category in the data set; wherein the graph structure information includes graph node IDs and edge connection information; the number of nodes K that need to be expanded is given; the node pseudo labels are given, and the initial pseudo label matrix is ​​an n×n unit matrix;

[0043] Step 12, the initial pseudo label matrix and the adjacency matrix are iterated cyclically to obtain a new pseudo label matrix PL;

[0044] Step 13, pre-training the E step: input the graph structure information, node attributes, and pseudo-label matrix PL to pre-train the GNN model; obtain the pre-trained GNN model and the predicted labels of the unknown label nodes;

[0045] Step 14, perform the first M-step training: input the graph structure information, the pseudo-label matrix PL, and use the predicted labels of the unknown label nodes as the true labels of the loss function, train the GNN model to obtain the training model and predicted labels after the first M-step training;

[0046] Perform the first E-step training: Use the predicted label after the first M-step training as the true label of the loss function of the first E-step training, input the graph structure information and node features, perform E-step training on the pre-trained GNN model, and obtain the predicted label after the first E-step training;

[0047] Perform the second M-step training: Use the predicted label after the first E-step training as the true label of the loss function of the second M-step training, and input the graph structure information and pseudo label matrix PL. Perform the second M-step training on the model after the first M-step training to obtain the predicted label of the second M-step training and the attention coefficient of the pseudo label. The collection of pseudo labels here has an attention coefficient Obtained from the second layer of the GAT model in the GNN model;

[0048] Perform the second E-step training: Use the predicted label after the second M-step training as the true label of the loss function of the second E-step training, input the graph structure information and node features, perform the second E-step training on the model after the first E-step training, and obtain the attention coefficient of the node features of the second E-step training A collection of node features, where the attention coefficient is It can be obtained in the second layer GAT model in the GNN model.

[0049] Furthermore, the probability matrix calculation module is specifically used to implement the following process:

[0050] Step 21, use formula (11) to calculate the mean attention coefficient E of node i and neighbor node j regarding node features and pseudo labels: ij :

[0051]

[0052] Step 22: Use formula (12) to calculate the mean sum of attention coefficients SE of node i and all neighboring nodes regarding features and pseudo labels:i :

[0053]

[0054] Step 23, use formula (13) to calculate the probability coefficient of node i wandering to neighbor node j; and combine the probability coefficients of all nodes and their neighbor nodes into a probability matrix according to the adjacency matrix of the graph

[0055]

[0056] Furthermore, the node expansion module is specifically used to implement the following process:

[0057] Step 31, the probability matrix A obtained in step 2 is used as a probability transfer matrix, and a personalized PageRank algorithm is used to process the probability matrix A to obtain the Pr value of each node;

[0058] Step 32, sort each node Pr outside the obtained seed set in descending order, and extract K / T nodes with the highest Pr value; K is the number of nodes to be expanded, which is a positive integer, and T is the number of cycles of the expansion process;

[0059] Step 33, adding the obtained nodes to the seed set to obtain a new seed set;

[0060] Step 34, determine whether the number of loops reaches T times, if yes, execute step 35, otherwise return to step 31; after T loops, a total of K+|S| nodes are obtained;

[0061] Step 35: remove the nodes in the initial seed set from the obtained nodes, a total of |S| nodes, and obtain the node set N consisting of K nodes expanded to K , the expansion ends.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] 1. The present invention first trains two graph neural networks through the EM algorithm. The E-step and the M-step cooperate with each other. It is expected that the label predicted by the graph neural network in the E-step is similar to the label predicted by the graph neural network in the M-step. The training is stopped when the node labels predicted by the E-step and the M-step are very similar. At this time, the best node feature representation can be learned and the most accurate node label can be predicted. The corresponding attention coefficient is calculated through the best node feature representation and node label. At this time, the attention coefficient can more accurately represent the relationship between the node and the neighboring node.

[0064] 2. In the GNN model trained in the E-step and M-step of the EM algorithm of the present invention, the GAT model and the fully connected layer are mainly used. The GAT model can obtain a powerful node feature representation. In the process of learning the feature representation of any node i, the GAT model calculates the attention coefficient of node i and its neighboring nodes and the attention coefficient pointing to node i itself, which can more accurately represent the relationship between the node and the neighboring nodes. According to the attention coefficient, the features of the neighboring nodes of node i and the features of node i itself are aggregated to obtain a new feature representation of node i, and then the node label is predicted through the fully connected layer.

[0065] 3. The probability matrix A is calculated based on the attention coefficients of the pseudo-labels and the node features. In this process, the attention coefficient of node i pointing to itself is first subtracted, and the attention coefficients of any node i and its neighboring nodes in the graph are normalized to obtain the probability coefficient of node i wandering to the neighboring node. The probability coefficients constitute a probability matrix, which provides a biased wandering method for subsequent personalized PageRank wandering.

[0066] 4. In the personalized PageRank algorithm, the probability transfer matrix provides a random walk method. During the walk, the probability of node i walking to any neighbor node j is equal. In the present invention, the probability matrix obtained in the previous step is used as the probability transfer matrix. For any node i, the probability of walking to node j is different. It is substituted into the personalized PageRank algorithm to perform a biased walk, so as to more accurately expand according to the seed set and obtain better expansion results.

[0067] In summary, the present invention combines labels and node attribute information and applies them in the seed set expansion process. Experiments show that compared with the traditional seed set expansion algorithm, the expanded node set has more nodes belonging to the community where the seed set is located. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a flow chart of the seed set expansion method based on graph neural network of the present invention.

[0069] Figure 2 (a) is the result of comparing the recall rate of the proposed method with the traditional seed set expansion algorithm under different K values ​​and randomly selected seed sets for the dataset PubMed.

[0070] Figure 2 (b) is the result of comparing the recall rate of the proposed method with the traditional seed set expansion algorithm when the CiteSeer dataset has different K values ​​and randomly selects seed sets.

[0071] Figure 3(a) is the result of comparing the recall rate of the proposed method with the traditional seed set expansion algorithm when the dataset CiteSeer has different K values ​​and preferably selects nodes with high selectivity as the seed set.

[0072] Figure 3 (b) is the result of comparing the recall rate of the proposed method with the traditional seed set expansion algorithm for the dataset Cora under different K values ​​and with nodes with high selectivity being preferred as the seed set.

[0073] Figure 4 (a) is the result of comparing the number of nodes expanded to the same community by the proposed method and the traditional seed set expansion algorithm when the dataset CiteSeer has different K values ​​and randomly selects seed sets.

[0074] Figure 4 (b) is the result of comparing the number of nodes expanded to the same community by the proposed method and the traditional seed set expansion algorithm when the dataset Cora has different K values ​​and randomly selected seed sets.

[0075] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. DETAILED DESCRIPTION

[0076] like Figure 1 As shown, the seed set expansion method (gnn-pagerank) based on graph neural network of the present invention comprises the following steps:

[0077] Step 1: For the given graph data, obtain the graph structure information and node attributes. Given the pseudo-label, seed set and the number of nodes K to be expanded, train two graph neural networks through the EM algorithm to calculate the attention coefficient of the pseudo-label and the attention coefficient of the node feature.

[0078] The specific operations are as follows:

[0079] Step 11, obtain graph structure information, node attributes, pseudo labels, seed sets, and the number of nodes K that need to be expanded. Among them, the seed set S is randomly obtained. For example, the nodes in the data set CiteSeer belong to 6 categories, among which there are 224 nodes of category 0. The seed set ratio is set to 10% of the category to which it belongs. The seed set ratio is multiplied by the number of nodes of the category to which the seed set belongs, which is 22 (rounded down). Then 22 nodes of category 0 are randomly selected as the seed set (other categories can also be set, see the embodiment of the present invention. That is, the seed set is obtained by taking a certain proportion of nodes from the nodes of the same category in the data set), and the seed set is nodes with known labels; wherein the graph structure information includes the graph node ID and the connection information of the edge, which is a known data set; given the number of nodes K to be expanded, the number of nodes K of different data sets is different; given the node pseudo-label, the initial pseudo-label matrix is ​​an n×n unit matrix, each row of the pseudo-label matrix is ​​the pseudo-label of a node, n is the number of nodes, the pseudo-label of the i-th node is represented by a row vector with the i-th column as 1 and the other columns as 0. After this setting, the initial pseudo-label matrix is ​​represented as a unit matrix.

[0080] Step 12, loop iteratively process the initial pseudo label matrix and the adjacency matrix to obtain a new pseudo label matrix PL. This includes the following operations:

[0081] Step 1: Use the initial pseudo-label matrix as the preds matrix;

[0082] Step 2: Use the result obtained by multiplying the current preds matrix with the adjacency matrix to replace the current preds matrix; then, use the result obtained by adding the current preds matrix to the pseudo-label matrix to replace the current preds matrix;

[0083] Step 3: After setting the values ​​of the elements in the current preds matrix output in the second step that are greater than 1 to 1, the resulting matrix is ​​used as the current preds matrix;

[0084] Step 4: Repeat steps 2 and 3 for a total of 2 times (i.e., step 2 and step 3 in step 12 are performed 3 times in total), assign the preds matrix outputted last time to the pseudo-label matrix, and obtain a new pseudo-label matrix PL;

[0085] Step 13, pre-training the E step: input the graph structure information and node attributes, pseudo label matrix PL (obtained from step 2), pre-train the GNN model (train 200 times); obtain the pre-trained GNN model and the predicted labels of the unknown label nodes;

[0086] Step 14, perform the first M-step training: input the graph structure information, the pseudo-label matrix PL (obtained from step 2), and use the predicted label of the unknown label node as the true label of the loss function, train the GNN model (train 100 times) to obtain the training model and predicted label after the first M-step training;

[0087] Perform the first E-step training: Use the predicted label after the first M-step training as the true label of the loss function of the first E-step training, input the graph structure information and node features (i.e. node attributes), perform E-step training (100 times) on the pre-trained GNN model, and obtain the predicted label after the first E-step training;

[0088] Perform the second M-step training: Use the predicted label after the first E-step training as the true label of the loss function of the second M-step training, and input the graph structure information and pseudo label matrix PL. Perform the second M-step training (100 times) on the model after the first M-step training to obtain the predicted label of the second M-step training and the attention coefficient of the pseudo label. The collection of pseudo labels here has an attention coefficient Obtained from the second layer of the GAT model in the GNN model;

[0089] Perform the second E-step training: Use the predicted label after the second M-step training as the true label of the loss function of the second E-step training, input the graph structure information and node features, perform the second E-step training (100 times) on the model after the first E-step training, and obtain the attention coefficient of the node features of the second E-step training A collection of node features, where the attention coefficient is It can be obtained in the second layer GAT model in the GNN model.

[0090] In step 13 and step 14, in the GNN model, the first layer is a GAT model, the second layer is also a GAT model, and the third layer is a fully connected layer.

[0091] Step 1 trains two graph neural networks through the EM algorithm. The goal of the E-step training is to calculate the attention coefficient of the node feature in the process of predicting the node label. The goal of the M-step training is to calculate the attention coefficient about the label by learning the node labels of the neighboring nodes. In the EM algorithm process, the label predicted in the E-step is used as the true label for the M-step to predict the label. The predicted label output by the M-step is provided to the E-step. The E-step and the M-step cooperate with each other. It is expected that the label predicted by the E-step is similar to the label predicted by the M-step. The E-step and the M-step stop training when the node labels are very similar, and learn the best node feature representation and node label. In addition, in the GNN model trained in the E-step and the M-step, the first layer is the GAT model, the second layer is the GAT model, and the third layer is the fully connected layer. The GAT model can obtain powerful feature representation learning, and finally obtain the attention coefficient between the node and the neighboring node, which can more accurately represent the relationship between the node and the neighboring node.

[0092] Step 2: Calculate the probability matrix A based on the attention coefficient of the pseudo-label and the attention coefficient of the node feature. The specific operation is as follows:

[0093] Step 21, use formula (11) to calculate the mean attention coefficient E of node i and neighbor node j regarding node features and pseudo labels: ij :

[0094]

[0095] Step 22: Use formula (12) to calculate the mean sum of attention coefficients SE of node i and all neighboring nodes regarding features and pseudo labels: i :

[0096]

[0097] Step 23, use formula (13) to calculate the probability coefficient of node i wandering to neighbor node j; and combine the probability coefficients of all nodes and their neighbor nodes into a probability matrix according to the adjacency matrix of the graph

[0098]

[0099] Step 3: Use the probability coefficient to guide PageRank to wander and obtain the node to which it is expanded.

[0100] The specific operations are as follows:

[0101] Assume that the number of nodes to be expanded is K, and divide the expansion process into T times (T is a positive integer that makes K divisible by T), that is, loop T times, and the process of each loop is as follows:

[0102] Step 31, the probability matrix A obtained in step 2 is used as a probability transfer matrix, and a personalized PageRank algorithm is used to process it to obtain the Pr value of each node.

[0103] Specifically, in the personalized PageRank algorithm, the Pr value of the seed set is set to 1, the Pr values ​​of the remaining nodes are set to 0, and the damping coefficient d is set to 0.85.

[0104] Step 32, sort each node Pr outside the obtained seed set in descending order, and select K / T nodes with the highest Pr value. (K is a positive integer, and the value of K in each instance is different)

[0105] Step 33, adding the obtained nodes to the seed set to obtain a new seed set.

[0106] Step 34, determine whether the number of loops reaches T times, if yes, execute step 35, otherwise return to step 31; after T loops, a total of K+|S| nodes are obtained;

[0107] Step 35, remove the nodes in the initial seed set (|S| in total) from the obtained nodes, and obtain the node set N consisting of the expanded K nodes. K , the expansion ends.

[0108] In order to verify the feasibility and effectiveness of the present invention, the present invention tested the given model through multiple groups of experiments, and used relative recall to evaluate the extension effect of the method of the present invention and the PageRank algorithm. The networkx version used in the experiment is 2.3, and the compilation language is python3.6. The computer processor used in the experiment is Genuine Intel (R) CPU @ 2.90GHz, the memory is 64G, and the operating system is Windows10.

[0109] Related terms: Recall: It is a measure of coverage, measuring how many positive examples are classified as positive examples, recall = TP / (TP+FN) = TP / P. In the seed set expansion problem, recall is divided into absolute recall and relative recall. The relative recall of seed set expansion is |P∩U| / |CS|, and the absolute recall is |P∩C| / |C|. The relative recall is the proportion of nodes belonging to the seed set community to all nodes in the seed set community minus the seed set. The absolute recall is the proportion of nodes belonging to the seed set community to all nodes in the seed set community.

[0110] In all instances, the relevant parameters are set as follows: the learning rate of the graph neural network is 0.005, the optimizer used in the training process is Adam SGD, the dimension of the intermediate hidden layer is 8, and the weight decay is 0.0005. In the E-step and M-step graph neural networks, the input values ​​(node ​​features in the E-step and pseudo labels in the M-step) are first processed with the dropout function. The probability of the neuron being selected in the dropout function is set to 0.6, and the feature value is obtained by processing. Then the GAT model is connected, the multi-head attention nheads is set to 8, and then the activation function relu is used for processing, and then the droput function is connected. The probability of the neuron being selected in the dropout function is still set to 0.6, and then the second GAT layer is used, nheads is set to 1, and then the relu activation function is used for processing. Finally, the two parameters of the fully connected layer are set to the hidden layer dimension 8 and the number of data set categories classes respectively.

[0111] The datasets used in the embodiments are PubMed, CiteSeer, and Cora. ​​All datasets are citation network datasets and are public datasets.

[0112] Dataset PubMed Cora CiteSeer Classes 3 7 6 Nodes 19717 2708 3327 Edges 44338 5429 4732 Features 500 1433 3703

[0113] Embodiment 1:

[0114] All nodes with the same label in the data set are grouped into one community. This embodiment uses the data sets PubMed and CiteSeer. The data set PubMed has three categories, and K = [400, 800, 1200, 1600, 2000, 2400, 2800] is taken respectively. The seed set is randomly selected for the three categories 0, 1, and 2 for 5 times (a total of 15 times), and the percentage of seed set selection is |S| / |C| = 0.1 (the number of nodes in the seed set accounts for the number of nodes in the category); T is taken 10 times (for different K, it is expanded 10 times), the recall rate is calculated and the average is taken to obtain the recall rate of the data set PubMed under different K. The gnn-pagerank algorithm of the present invention is the recall rate calculated using the node attributes. The results are as follows: Figure 2 As shown in a, the recall rate of the gnn-pagerank algorithm is significantly higher than that of the traditional algorithms pagerank and dn-pagerank. For the CiteSeer dataset, there are 6 categories. For each category, K = [40, 80, 120, 160, 200, 240, 280] is taken. For the 6 categories 0, 1, 2, 3, 4, and 5, the seed set is randomly selected 5 times (30 times in total). The seed set selection ratio is |S| / |C| = 0.1, T is 10 (for each K, it is expanded 10 times), the recall rate is calculated and the average is taken to obtain the recall rate of the CiteSeer dataset under different K. The results are shown in Figure 2 As shown in (b), the recall rate of the GNN-PageRank algorithm is significantly higher than that of the traditional seed set expansion algorithms PageRank and DN-PageRank.

[0115] Embodiment 2:

[0116] Select nodes with high degree as the seed set, set all nodes with the same label as a community, and select all nodes with high degree in the community as the seed set. The CiteSeer dataset has 6 categories. Sort the nodes in each category in descending order according to their degree. Select the top 10% of the nodes in each class as the seed set. Take K = [100, 200, 300, 400, 500], T is 10, and expand each K 10 times to obtain the expansion results of each class under different K conditions. Calculate the recall rate of each class and take the average recall rate of the 6 classes under different K conditions. The results are as follows: Figure 3 As shown in (a), the recall rate of the gnn-pagerank algorithm of the present invention is significantly higher than that of the traditional seed set expansion algorithm, that is, the expansion accuracy is higher than that of the traditional seed set expansion algorithm. The Cora dataset has 7 categories. The nodes in each category are sorted in descending order according to their degrees. The top 10% of the nodes in each category are selected as the seed set. K = [100, 200, 300, 400, 500, 600], T is 10, and each K is expanded 10 times to obtain the expansion results of each category under different K conditions. The recall rate of each category under different K conditions is calculated and the average is taken. The results are shown in Figure 3 As shown in (b), when K is greater than 200, the recall rate of the gnn-pagerank algorithm is slightly higher than that of the traditional seed set expansion algorithm dn-pagerank, and much higher than that of the traditional seed set expansion algorithm pagerank.

[0117] Embodiment 3:

[0118] Set all nodes with the same label as a community, randomly obtain seed sets, the proportion of seed sets in the same community is 0.3, and compare the number of nodes found. For the CiteSeer dataset, there are 6 categories. For categories 0, 1, 2, 3, 4, and 5, K = [40, 80, 120, 160, 200, 240, 280] are taken respectively. Under different K conditions, the seed sets are randomly selected 5 times for each of the 6 categories (30 times in total), and the average is taken and used as the number of nodes obtained by different K expansions. T is 10 (for each K, it is divided into 10 times for expansion). The number of nodes obtained by expansion under different K conditions is as follows: Figure 4As shown in a, it is obvious that the number of nodes expanded by the gnn-pagerank algorithm of the present invention is greater than that of the traditional seed set expansion algorithm pagerank and dn-pagerank algorithms. Similarly, for the data set PubMed, there are 3 categories. For categories 0, 1, and 2, K=[400, 800, 1200, 1600, 2000, 2400, 2800, 3200] are taken respectively. Under different K conditions, the seed sets of the three classes are randomly selected 5 times (30 times in total), and the average is taken, and it is used as the number of nodes expanded with different K. T is taken as 10 (for each K, it is expanded 10 times), and the number of nodes expanded under different K conditions is shown as follows Figure 4 As shown in b, it is obvious that the number of nodes expanded by the PageRank algorithm is much lower than that of the GNN-PageRank algorithm, and the number of nodes expanded by the Dn-PageRank algorithm is slightly lower than that of the GNN-PageRank algorithm.

[0119] In summary, the method (GNN-PageRank) proposed in the present invention is superior to the traditional seed set expansion algorithm in most cases in terms of the recall rate and the number of nodes obtained by expansion under different conditions of given seed sets in the problem of seed set expansion.

Claims

1. A seed set expansion method based on graph neural network, characterized in that: The specific steps include: Step 1: For the given graph data, obtain the graph structure information and node attributes, and given the pseudo-label, seed set, and the number of nodes to be expanded K, train two graph neural networks through the EM algorithm to calculate the attention coefficient of the pseudo-label and the attention coefficient of the node feature; Step 2: Calculate the probability matrix A based on the attention coefficient of the pseudo-label and the attention coefficient of the node feature; Step 3: Expand the number of nodes K as needed, use the probability coefficients in the probability matrix A to guide PageRank to wander, and obtain the nodes that are expanded; The step 1 specifically includes the following sub-steps: Step 11, obtain graph structure information, node attributes, pseudo labels, seed set, and the number of nodes K that need to be expanded; the seed set is obtained by taking a certain proportion of nodes of the same category in the data set; wherein the graph structure information includes graph node IDs and edge connection information; given the number of nodes that need to be expanded K ; Given a node pseudo label, the initial pseudo label matrix is ​​an n×n unit matrix; the seed set is obtained by taking a certain proportion of nodes of the same category in the data set, the data set is a citation network data set, and the data set is PubMed, CiteSeer or Cora; Step 12, the initial pseudo label matrix and the adjacency matrix are iterated cyclically to obtain a new pseudo label matrix PL; Step 13, pre-training the E step: input the graph structure information, node attributes, and pseudo-label matrix PL to pre-train the GNN model; obtain the pre-trained GNN model and the predicted labels of the unknown label nodes; Step 14, perform the first M-step training: input the graph structure information, the pseudo-label matrix PL, and use the predicted labels of the unknown label nodes as the true labels of the loss function, train the GNN model to obtain the training model and predicted labels after the first M-step training; Perform the first E-step training: Use the predicted label after the first M-step training as the true label of the loss function of the first E-step training, input the graph structure information and node features, perform E-step training on the pre-trained GNN model, and obtain the predicted label after the first E-step training; Perform the second M-step training: Use the predicted label after the first E-step training as the true label of the loss function of the second M-step training, and input the graph structure information and pseudo label matrix PL. Perform the second M-step training on the model after the first M-step training to obtain the predicted label of the second M-step training and the attention coefficient of the pseudo label. The collection of pseudo labels here has an attention coefficient Obtained from the second layer of the GAT model in the GNN model; Perform the second E-step training: Use the predicted label after the second M-step training as the true label of the loss function of the second E-step training, input the graph structure information and node features, perform the second E-step training on the model after the first E-step training, and obtain the attention coefficient of the node features of the second E-step training A collection of node features, where the attention coefficient is It can be obtained from the second layer GAT model in the GNN model; The step 2 specifically includes the following sub-steps: Step 21, use formula (11) to calculate the node With neighbor nodes The mean of the attention coefficients for node features and pseudo labels : （11）； Step 22, use formula (12) to calculate the node The mean sum of attention coefficients of all neighboring nodes on features and pseudo labels : （12）； Step 23, use formula (13) to calculate the node Walk to neighbor nodes The probability coefficient of each node and its neighboring nodes is formed into a probability matrix according to the adjacency matrix of the graph. ; (13)。 2. The seed set expansion method based on graph neural network according to claim 1, characterized in that: The step 12 specifically includes the following operations: Step 1: Use the initial pseudo-label matrix as the preds matrix; Step 2: Use the result obtained by multiplying the current preds matrix with the adjacency matrix to replace the current preds matrix; then, use the result obtained by adding the current preds matrix to the pseudo-label matrix to replace the current preds matrix; Step 3: After setting the values ​​of the elements in the current preds matrix output in the second step that are greater than 1 to 1, the resulting matrix is ​​used as the current preds matrix; Step 4: Repeat steps 2 and 3 for a total of 2 times, that is, step 2 and step 3 in step 12 are performed 3 times in total, and the preds matrix outputted last time is assigned to the pseudo-label matrix to obtain a new pseudo-label matrix PL.

3. The seed set expansion method based on graph neural network according to claim 1, characterized in that: In the GNN model in step 13 and step 14, the first layer is a GAT model, the second layer is also a GAT model, and the third layer is a fully connected layer.

4. The seed set expansion method based on graph neural network according to claim 1, characterized in that: The step 3 specifically includes the following sub-steps: Step 31, the probability matrix A obtained in step 2 is used as a probability transfer matrix, and a personalized PageRank algorithm is used to process the probability matrix A to obtain the PR value of each node; Step 32: sort the PR of each node outside the obtained seed set in descending order, and take out the node with the highest PR value. nodes; K is the number of nodes that need to be expanded, which is a positive integer. is the number of cycles of the expansion process; Step 33, adding the obtained nodes to the seed set to obtain a new seed set; Step 34, determine whether the number of cycles has reached T If yes, execute step 35, otherwise return to step 31; loop After that, we got nodes; Step 35, remove the nodes in the initial seed set from the obtained nodes, a total of , and get the node set consisting of K nodes expanded to , the expansion ends.

5. A seed set expansion system based on graph neural network, characterized in that: Used to implement the method described in any one of claims 1 to 4, specifically comprising the following modules: The attention coefficient calculation module is used to implement the following functions: for given graph data, obtain graph structure information and node attributes, given pseudo labels, seed sets, and the number of nodes to be expanded K, train two graph neural networks through the EM algorithm, and calculate the attention coefficients of pseudo labels and node features; A probability matrix calculation module is used to calculate the probability matrix A according to the attention coefficient of the pseudo label and the attention coefficient of the node feature; The node expansion module is used to expand the number of nodes K as needed, use the probability coefficients in the probability matrix A to guide PageRank to walk, and obtain the expanded nodes; The attention coefficient calculation module is specifically used to implement the following process: Step 11, obtain graph structure information, node attributes, pseudo labels, seed set, and the number of nodes K that need to be expanded; the seed set is obtained by taking a certain proportion of nodes of the same category in the data set; wherein the graph structure information includes graph node IDs and edge connection information; given the number of nodes that need to be expanded K ; Given a node pseudo label, the initial pseudo label matrix is ​​an n×n identity matrix; Step 12, the initial pseudo label matrix and the adjacency matrix are iterated cyclically to obtain a new pseudo label matrix PL; Step 13, pre-training the E step: input the graph structure information, node attributes, and pseudo-label matrix PL to pre-train the GNN model; obtain the pre-trained GNN model and the predicted labels of the unknown label nodes; Step 14, perform the first M-step training: input the graph structure information, the pseudo-label matrix PL, and use the predicted labels of the unknown label nodes as the true labels of the loss function, train the GNN model to obtain the training model and predicted labels after the first M-step training; Perform the first E-step training: Use the predicted label after the first M-step training as the true label of the loss function of the first E-step training, input the graph structure information and node features, perform E-step training on the pre-trained GNN model, and obtain the predicted label after the first E-step training; Perform the second M-step training: Use the predicted label after the first E-step training as the true label of the loss function of the second M-step training, and input the graph structure information and pseudo label matrix PL. Perform the second M-step training on the model after the first M-step training to obtain the predicted label of the second M-step training and the attention coefficient of the pseudo label. The collection of pseudo labels here has an attention coefficient Obtained from the second layer of the GAT model in the GNN model; Perform the second E-step training: Use the predicted label after the second M-step training as the true label of the loss function of the second E-step training, input the graph structure information and node features, perform the second E-step training on the model after the first E-step training, and obtain the attention coefficient of the node features of the second E-step training A collection of node features, where the attention coefficient is It can be obtained from the second layer GAT model in the GNN model; The probability matrix calculation module is specifically used to implement the following process: Step 21, use formula (11) to calculate the node With neighbor nodes The mean of the attention coefficients for node features and pseudo labels : （11）； Step 22, use formula (12) to calculate the node The mean sum of attention coefficients of all neighbor nodes on features and pseudo labels : （12）； Step 23, use formula (13) to calculate the node Walk to neighbor nodes The probability coefficient of each node and its neighboring nodes is formed into a probability matrix according to the adjacency matrix of the graph. ; (13)。 6. The seed set expansion system based on graph neural network according to claim 5, characterized in that: The node expansion module is specifically used to implement the following process: Step 31, the probability matrix A obtained in step 2 is used as a probability transfer matrix, and a personalized PageRank algorithm is used to process the probability matrix A to obtain the Pr value of each node; Step 32: for each node outside the obtained seed set Sort in descending order and take out The highest value nodes; K is the number of nodes that need to be expanded, which is a positive integer. is the number of cycles of the expansion process; Step 33, adding the obtained nodes to the seed set to obtain a new seed set; Step 34, determine whether the number of cycles has reached T If yes, execute step 35, otherwise return to step 31; loop After that, we got nodes; Step 35, remove the nodes in the initial seed set from the obtained nodes, a total of , and get the node set consisting of K nodes expanded to , the expansion ends.

Images