Multi-model unbalanced node classification method and device

Abstract

The embodiment of the invention provides a multi-model unbalanced node classification method and device. The method comprises the following steps: acquiring a graph data set, wherein the graph data set comprises a training set and a test set; generating a synthesis node based on the information of each node in the training set, so that the number of nodes of each category in the training set is distributed in a balanced manner; adding all the generated synthesis nodes into the training set to form a new training set, dividing the new training set into a plurality of training subsets, and respectively inputting the training subsets into a plurality of sub-models in a graph neural network model for training until the performance of the graph neural network model is converged; and inputting the test set into the performance convergent graph neural network model for node classification. According to the method and the device provided by the embodiment of the invention, the node classification performance can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of graph processing, and specifically to a multi-model unbalanced node classification method and device. Background Art

[0002] Graph Neural Network (GNN) is a framework that has emerged in recent years and uses deep learning to directly learn graph structured data. By formulating certain strategies on the nodes and edges in the graph, GNN converts graph structured data into a standardized and standard representation, and inputs it into a variety of different neural networks for training, achieving excellent results in tasks such as node classification.

[0003] Node classification is considered an important task in graph analysis. With the rapid development of neural networks in recent years, GNN has become the de facto standard for handling this task and has achieved remarkable performance in many graph-related application scenarios. Current GNNs are implicitly based on the class balance assumption, that is, the number of training instances of all classes is roughly balanced, and class imbalance refers to the number of instances of some classes being much smaller than other classes. The class imbalance problem is prevalent in real-world applications of various machine learning tasks. Since the number of samples of the majority class is much larger than that of the minority class in the training set, it is easy for machine learning models to underestimate the minority class, which leads to poor overall classification results. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a multi-model unbalanced node classification method and device, which can improve the performance of node classification.

[0005] In a first aspect, a multi-model unbalanced node classification method is provided, comprising: obtaining a graph data set, the graph data set comprising a training set and a test set; generating a synthetic node based on information of each node in the training set, so that the number of nodes in each category in the training set is evenly distributed, comprising the following steps: calculating a predicted probability vector and a uniform distribution of each node in the training set, and calculating a total variation distance between the predicted probability vector and the uniform distribution of each node, and using it as an uncertainty score of each node; processing the uncertainty score of each node based on the following formula to obtain a node risk value of each node: Among them, U Θ (v) is the uncertainty score of node v, is the number of nodes of category j, is the category predicted by the model The number of nodes in the training set, C is the total number of categories in the training set; according to the node risk value of each node, a key node is determined from the training set, and the key node is a node that is difficult to be correctly classified in the training set; the node directly connected to the key node in the training set is determined as an auxiliary node; the feature of the synthetic node is generated according to the following formula: syn =δX anc +(1-δ)X aux , δ is a random value between 0 and 1, X syn is the characteristic of the synthesis node, X anc is the feature of the key node, X aux is a feature of the auxiliary node; sampling from the one-hop subgraph of the key node to generate the edge of the synthetic node; and using the graph diffusion smoothing technique to convert the diffusion matrix S of the training set into a weighted and sparse adjacency matrix And use the adjacency matrix As the probability of polygon distribution, neighbor nodes are sampled from the one-hop subgraph of the key node to form an adjacency set of the synthetic node; all the generated synthetic nodes are added to the training set to form a new training set, and the new training set is divided into multiple training subsets and respectively input into multiple sub-models in the graph neural network model for training until the performance of the graph neural network model converges. For each training subset in the multiple training subsets, the training process includes the following steps: calculating the neighbor label distribution of each node in each training subset and the category connection matrix of each category in the total categories corresponding to each training subset; calculating the abnormal connection perception edge ACM of any node in each training subset on any category in the total categories corresponding to each training subset according to the following formula: in, is the ACM value of node v in category t, is the category y to which node v belongs v The average degree of internal node connections, is a neighbor node of node v that belongs to category y v The ratio of D v，t is the proportion of node v’s neighbor nodes belonging to category t, is category y v The ratio of the average connection between class t and class t; the loss function of the sub-model corresponding to each sub-training set is calculated according to the following formula: L TAM is the loss function, is the logits of node v, y v is the category of node v, Represents the ACM term of node v, α is a hyperparameter; based on the loss function of the sub-model corresponding to each sub-training set, optimize each sub-model separately; and for each sub-model in the multiple sub-models, after each training using the above steps, remove the correctly classified nodes in the sub-training set corresponding to each sub-model, and add the nodes in the sub-training sets corresponding to other sub-models in the multiple sub-models to perform the above training steps again until all nodes in the new training set are correctly classified; input the test set into the graph neural network model with converged performance for node classification.

[0006] In a possible implementation, calculating the total variation distance between the predicted probability vector of each node and the uniform distribution and taking the total variation distance as the uncertainty score of each node includes obtaining the uncertainty score of each node based on the following formula: in, is the predicted probability vector of node v The node predicted by the model belongs to the category The uniform distribution of is the predicted probability that node v belongs to category j, is the indicator function, when 1 if the value is 0, otherwise it is 0.

[0007] In a possible implementation manner, the multi-model unbalanced node classification method further includes: sampling from a random distribution to obtain a random value δ.

[0008] In a possible implementation, the multi-model unbalanced node classification method further includes: evaluating whether the graph neural network model converges based on the accuracy and F1 score of the training results.

[0009] The multi-model unbalanced node classification method also includes: the graph neural network model includes at least one of a graph convolutional network GCN, a graph attention network GAT, and a graph sampling and aggregation GraphSAGE.

[0010] In a second aspect, a multi-model unbalanced node classification device is provided, including: an acquisition module, used to acquire a graph data set, the graph data set including a training set and a test set; a synthetic node generation module, used to generate synthetic nodes based on the information of each node in the training set, so that the number of nodes in each category in the training set is evenly distributed, and the synthetic node generation module is specifically used to: calculate the predicted probability vector and uniform distribution of each node in the training set, and calculate the total variation distance between the predicted probability vector and the uniform distribution of each node, and use it as the uncertainty score of each node, and process the uncertainty score of each node based on the following formula to obtain the node risk value of each node: Among them, U Θ (v) is the uncertainty score of node v, is the number of nodes of category j, is the category predicted by the model The number of nodes in the training set, C is the total number of categories in the training set, and according to the node risk value of each node, a key node is determined from the training set, and the key node is a node that is difficult to be correctly classified in the training set. The nodes directly connected to the key node in the training set are determined as auxiliary nodes, and the features of the synthetic nodes are generated according to the following formula: X syn =δX anc +(1-δ)X aux , δ is a random value between 0 and 1, X syn is the characteristic of the synthesis node, X anc is the feature of the key node, X aux The feature of the auxiliary node is sampled from the one-hop subgraph of the key node to generate the edge of the synthetic node, and the diffusion matrix S of the training set is converted into a weighted and sparse adjacency matrix using graph diffusion smoothing technology And use the adjacency matrix As the probability of polygon distribution, neighbor nodes are sampled from the one-hop subgraph of the key node to form an adjacency set of the synthetic node; a training module is used to add all the generated synthetic nodes to the training set to form a new training set, and the new training set is divided into multiple training subsets and respectively input into multiple sub-models in the graph neural network model for training until the performance of the graph neural network model converges. For each training subset in the multiple training subsets, the training module is specifically used to: calculate the neighbor label distribution of each node in each training subset and the category connection matrix of each category in the total category corresponding to each training subset, and calculate the abnormal connection perception edge ACM value of any node in each training subset on any category in the total category corresponding to each training subset according to the following formula: in, is the ACM value of node v in category t, is the category y to which node v belongs v The average degree of internal node connections, is a neighbor node of node v that belongs to category y v The ratio of D v，t is the proportion of node v’s neighbor nodes belonging to category t, is category y v The ratio of the average connection between class t and class t is calculated according to the following formula to calculate the loss function of the sub-model corresponding to each sub-training set: L TAMis the loss function, is the logits of node v, y v is the category of node v, The ACM item represents the node v, α is a hyperparameter, and based on the loss function of the sub-model corresponding to each sub-training set, each sub-model is optimized respectively, and for each sub-model in the multiple sub-models, after each training using the above steps, the nodes correctly classified in the sub-training set corresponding to each sub-model are removed, and the nodes in the sub-training sets corresponding to other sub-models in the multiple sub-models are added to perform the above training steps again until all nodes in the new training set are correctly classified; a testing module is used to input the test set into the graph neural network model with converged performance for node classification.

[0011] In a possible implementation, the training module is specifically used to obtain the uncertainty score of each node based on the following formula: in, is the predicted probability vector of node v The node predicted by the model belongs to the category The uniform distribution of is the predicted probability that node v belongs to category j, is the indicator function, when 1 if the value is 0, otherwise it is 0.

[0012] In a possible implementation, the multi-model unbalanced node classification device further includes: a sampling module, configured to obtain a random value δ by sampling from a random distribution.

[0013] In a possible implementation, the multi-model unbalanced node classification device also includes: an evaluation module, used to evaluate whether the graph neural network model has converged based on the accuracy and F1 score of the training results.

[0014] In a possible implementation, the graph neural network model includes at least one of a graph convolutional network GCN, a graph attention network GAT, and a graph sampling and aggregation GraphSAGE.

[0015] In a third aspect, a multi-model unbalanced node classification device is provided, comprising: a processor, and a memory communicatively connected to the processor, wherein the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the multi-model unbalanced node classification method of the first aspect and any possible implementation method of the first aspect.

[0016] In a fourth aspect, a computer-readable storage medium is provided, which includes a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute a multi-model unbalanced node classification method such as the first aspect and any possible implementation method of the first aspect.

[0017] Based on the above technical solutions, the method of jointly generating minority class samples that are more difficult to classify and the method of adjusting the topology-aware boundary loss are combined, and the idea of ​​multi-model training is adopted to improve the performance of node classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the drawings without paying creative work.

[0019] Figure 1 A schematic block diagram of a multi-model unbalanced node classification method according to an embodiment of the present application is shown.

[0020] Figure 2 Another schematic block diagram of the multi-model unbalanced node classification method according to an embodiment of the present application is shown.

[0021] Figure 3 Another schematic block diagram of the multi-model unbalanced node classification method according to an embodiment of the present application is shown.

[0022] Figure 4 A schematic block diagram of a multi-model unbalanced node classification device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] GNN is a framework that has emerged in recent years and uses deep learning to directly learn graph structure data. By formulating certain strategies on the nodes and edges in the graph, GNN converts graph structure data into a standardized and standard representation and inputs it into a variety of different neural networks for training, achieving excellent results in tasks such as node classification.

[0025] Node classification is considered an important task in graph analysis. With the rapid development of neural networks in recent years, GNN has become the de facto standard for handling this task and has achieved remarkable performance in many graph-related application scenarios. Current GNNs are implicitly based on the class balance assumption, that is, the number of training instances of all classes is roughly balanced, and class imbalance refers to the number of instances of some classes being much smaller than other classes. The class imbalance problem is prevalent in real-world applications of various machine learning tasks. Since the number of samples of the majority class is much larger than that of the minority class in the training set, it is easy for machine learning models to underestimate the minority class, which leads to poor overall classification results.

[0026] Recent studies have found that countermeasures to alleviate the class imbalance problem can be roughly divided into two categories: loss modification and generative methods. Generative methods aim to augment the original class imbalance graph by synthesizing reasonable minority nodes to balance the class distribution, while loss modification methods aim to adjust the objective function to pay more attention to minority class samples.

[0027] However, existing generative methods cannot effectively expand the minority subspace to alleviate bias, while existing loss correction methods ignore the fact that some nodes may significantly degrade the performance of other classes when compensating the minority class based on quantity. Therefore, the current mitigation of class imbalance is still a problem that needs to be studied.

[0028] In view of this, an embodiment of the present application provides a multi-model unbalanced node classification method, a method for jointly generating minority class samples that are more difficult to classify, and a method for adjusting topology-aware boundary loss, and adopts the idea of ​​multi-model training to improve the performance of node classification.

[0029] Figure 1 A schematic block diagram of a multi-model unbalanced node classification method according to an embodiment of the present application is shown. Optionally, the multi-model unbalanced node classification method can be executed by a processor, which can be a general-purpose processor or a dedicated processor. For example, the processor can be a central processing unit (CPU). For another example, the processor can also be a graphics processing unit (GPU). Figure 1 As shown, the multi-model unbalanced node classification method 100 includes part or all of the following contents.

[0030] S110, obtaining a graph dataset, where the graph dataset includes a training set and a test set.

[0031] In some embodiments, the graph dataset may be a social network graph dataset, a traffic network graph dataset, a network security graph dataset, a knowledge graph dataset, or a financial network dataset, etc. For example, the graph dataset may be a Cora dataset.

[0032] Obtaining a graph dataset means obtaining relevant information of the graph dataset. For example, obtaining the node feature matrix X of the graph dataset, the adjacency matrix A of the graph dataset, or obtaining the diffusion matrix of the graph dataset. Usually, after obtaining the graph dataset, the graph dataset is divided into at least two parts, one as a training set and the other as a test set. In the embodiment of the present application, Marked as training set node, Y L is the label of the training set nodes; C is the total number of categories of the training set nodes.

[0033] S120, generating a synthetic node based on information of each node in the training set, so that the number of nodes of each category in the training set is evenly distributed.

[0034] Specifically, we can first clarify the category to which each node in the training set belongs, and also count the number of nodes in each category. Then, generate synthetic nodes for categories with insufficient number of nodes. For example, we can first calculate the maximum number of nodes in all categories (assuming it is N), and this maximum value is the target equilibrium number to be achieved for each category. For example, if the "cat" category has 100 nodes, which is the maximum, then the target equilibrium number N = 100. Then for categories with less than 100 nodes, we need to generate synthetic nodes to supplement them.

[0035] S130, adding all generated synthetic nodes to the training set to form a new training set, and dividing the new training set into multiple training subsets and inputting them into multiple sub-models in the graph neural network model for training respectively, until the performance of the graph neural network model converges.

[0036] Specifically, after the generation of the synthetic nodes is completed, these newly generated nodes need to be integrated into the original training set to form a new training set. Since the characteristics and structure of the synthetic nodes are consistent with the original nodes (for example, they meet the basic characteristics of graph data in terms of node attributes, edge connections, etc.), they can be directly added to the data structure of the training set. If the training set is stored in the form of a graph, for example, an adjacency matrix or adjacency table is used to represent the graph structure, then the information of the synthetic nodes (node ​​attributes, connection relationship with other nodes, etc.) needs to be accurately added to the adjacency matrix or adjacency table according to the corresponding data format requirements. Next, according to the specific needs and model structure, select a suitable method to divide the new training set into multiple training subsets. Common division methods include random division, proportional division, and division based on certain features. For example, the new training set can be randomly divided into three training subsets according to a certain ratio (such as 20%, 30%, 50%, etc.), or it can be divided into groups according to certain attributes of the nodes (such as the community to which the node belongs, the degree of the node, etc.). The graph neural network model can be composed of multiple sub-models, which can have the same or different structures and functions. For example, the multiple sub-models can be at least one of a Graph Convolutional Network (GCN), a Graph Attention Network (GAT), and a Graph Sampling and Aggregation (Graph SAGE). Then, the divided training subsets are input into the corresponding sub-models for training. During the training process, each sub-model will update and optimize parameters according to the input training subset data. The sub-model gradually adjusts its own parameters by learning the features of the nodes and edges in the training subset to improve the processing ability and prediction accuracy of the input data. During training, each sub-model will calculate the error between the prediction result and the true label according to its defined loss function (such as cross entropy loss, mean square error loss, etc.). Then, the gradient is calculated by the back-propagation algorithm, and the parameters of the model are updated according to the gradient, so that the loss function gradually decreases. This process will be repeated in multiple training rounds (epochs) until the performance of the sub-model on the training subset reaches a certain standard. If the performance indicators of the model no longer improve significantly, or the value of the loss function no longer decreases significantly in multiple consecutive training rounds, the performance of the model can be considered to have converged. When the model performance is judged to have converged, the training of the graph neural network model is stopped.

[0037] S140, inputting the test set into the performance-converged graph neural network model for node classification.

[0038] It should be noted that usually after selecting the graph neural network model, the graph neural network model can be initialized first.

[0039] In some embodiments, Figure 2 As shown, in S120, the following parts or all of the contents are included.

[0040] S121, calculating the prediction probability vector and uniform distribution of each node in the training set, and calculating the total variation distance between the prediction probability vector and the uniform distribution of each node, and using it as the uncertainty score of each node;

[0041] S122, processing the uncertainty score of each node based on the following formula to obtain a node risk value of each node: Among them, U Θ (v) is the uncertainty score of node v, is the number of nodes of category j, is the category predicted by the model The number of nodes, C is the total number of categories in the training set;

[0042] S123, determining a key node from the training set according to the node risk value of each node, wherein the key node is a node in the training set that is difficult to be correctly classified;

[0043] S124, determining the nodes in the training set that are directly connected to the key node as auxiliary nodes;

[0044] S125, generate the feature of the synthetic node according to the following formula: syn =δX anc +(1-δ)X aux , δ is a random value between 0 and 1, X syn is the characteristic of the synthesis node, X anc is the feature of the key node, X aux is the feature of the auxiliary node;

[0045] S126, sampling from the one-hop subgraph of the key node to generate the edge of the synthetic node;

[0046] S127, using graph diffusion smoothing technology to convert the diffusion matrix S of the training set into a weighted and sparse adjacency matrix And use the adjacency matrix As the probability of polygon distribution, neighbor nodes are sampled from the one-hop subgraph of the key node to form an adjacency set of the synthetic node.

[0047] In S121, each node in the training set is input into the initialized graph neural network model for prediction. The model outputs the probability distribution of each node belonging to different categories. For example, in a multi-classification task, there are C categories in total. For any node i in the training set, the model predicts the probability of the node belonging to each category to form a vector P i =(P i1 , P i2 ,......,P iC ), where P ij represents the predicted probability that node i belongs to the jth category, and This vector P i is the predicted probability vector of node i. For each node in the training set of C categories, its uniform distribution vector This is because a uniform distribution means that each class has an equal probability of being selected.

[0048] After obtaining the prediction probability vector and uniform distribution of each node, the total variation distance between the prediction probability vector and the uniform distribution of each node can be calculated based on the following steps and used as the uncertainty score of each node. That is, S121 can specifically include:

[0049] The uncertainty score of each node is obtained based on the following formula:

[0050] in, is the predicted probability vector of node v The node predicted by the model belongs to the category The uniform distribution of is the predicted probability that node v belongs to category j, is the indicator function, when 1 if the value is 0, otherwise it is 0.

[0051] The larger the total variation distance, the greater the difference between the predicted probability distribution of the node and the uniform distribution, which means that the model's prediction result for the node is relatively more certain; conversely, the smaller the total variation distance, the more uncertain the model's prediction result for the node. The total variation distance calculated for each node is directly used as the uncertainty score of the node. In this way, each node in the training set has a value representing its prediction uncertainty.

[0052] In S122, the uncertainty score of each node is processed to calculate the node risk value of each node. The uncertainty score reflects the uncertainty of the model's prediction of the node, and the node risk value further quantifies the impact of this uncertainty in different categories. In an embodiment of the present application, the calculation of the node risk value comprehensively considers the uncertainty score of the node and the proportional relationship between the number of nodes in different categories. In this way, the risk level of the node in the entire training set can be more comprehensively evaluated, which helps to further analyze the importance of the node or perform subsequent processing.

[0053] In S123, since the key node is a node that is difficult to be correctly classified in the training set, a standard is usually set to determine whether the node is a key node. This standard is generally related to the size of the node risk value. For example, a threshold value of the node risk value can be set. If the risk value of the node is greater than the threshold, the node is considered to be a key node. For another example, the risk values ​​of all nodes are sorted in order from large to small. Then a certain proportion of the top ranked nodes are selected as key nodes.

[0054] The data of the training set is usually presented in the form of a graph, where nodes represent data points and edges represent a certain connection relationship between nodes. For each node, there are other nodes connected to it. In the graph structure, an edge connects a pair of nodes, indicating that there is a direct relationship between the two nodes. In S124, for each key node, it is necessary to find the nodes directly connected to it. The nodes found to be directly connected to each key node are marked as auxiliary nodes, and the auxiliary node set is the set of all nodes directly connected to the key nodes.

[0055] In S125, first, a value δ is randomly generated in the range of 0 to 1. This value will determine the proportion of key node features and auxiliary node features in the composite node features. Key nodes are nodes that are previously determined to be difficult to be correctly classified in the training set. Each key node has its own feature vector X anc Auxiliary nodes are nodes directly connected to key nodes, and they also have their own eigenvectors X aux Substitute the randomly generated δ into the formula X syn =δX anc +(1-δ)X aux Calculation is performed. By generating the features of synthetic nodes in this way, the information of key nodes and auxiliary nodes can be integrated. Key nodes often represent the more difficult parts of the training set, while auxiliary nodes are directly related to key nodes and may contain relevant information that helps to process key nodes. The generation of synthetic nodes helps to further enrich data features and may help improve the model's ability to learn and process the relevant parts of these key nodes.

[0056] In some embodiments, the random value δ is sampled from a random distribution. For example, the random distribution can be a Beta distribution, which allows the algorithm to sample the δ value from the distribution, thereby introducing a certain amount of randomness and increasing the generalization ability of the model. By adjusting the parameters of the random distribution, the difficulty of synthesizing samples can be controlled to make them closer to key nodes or auxiliary nodes.

[0057] In S126, for each key node, its one-hop subgraph refers to a subgraph consisting of the key node and all its directly adjacent nodes (i.e., nodes directly connected by an edge). In graph theory, a one-hop neighbor node is a node with a distance of 1 from the key node, which can also be understood as a node directly connected to the key node. In some embodiments, the edges in the one-hop subgraph of the key node are randomly sampled. In other embodiments, if the edges in the one-hop subgraph have weights (for example, indicating the closeness of the connection between nodes, etc.), sampling can be performed according to the weights of the edges. Edges with higher weights have a greater probability of being selected as edges of synthetic nodes. After sampling, some edges in the one-hop subgraph are obtained as candidate edges. These candidate edges will be used to generate edges of synthetic nodes. The main purpose of sampling the edges of synthetic nodes from the one-hop subgraph of the key node is to enable it to inherit the structural information between the key node and its adjacent nodes when generating the synthetic node. This helps the synthetic node to be better integrated into the graph structure, and can reflect the local graph structural features around the key node, so that in subsequent operations such as graph neural network training, the model can better use this structural information for learning and prediction.

[0058] As can be seen from the above description, after obtaining the graph dataset, it is equivalent to obtaining various information of the graph dataset, including the diffusion matrix of the graph dataset. In graph analysis, the diffusion matrix usually represents the diffusion probability or propagation ability between nodes in the graph. For example, in a social network, the diffusion matrix may describe the possibility of information propagating from one user node to another user node. From the diffusion matrix of the obtained graph dataset, the diffusion matrix S of the training set can be extracted. In S127, the diffusion matrix S of the training set can be converted into a weighted and sparse adjacency matrix using graph diffusion smoothing technology. The graph diffusion smoothing technology can be, for example, an anisotropic diffusion filter algorithm or a random walk graph diffusion algorithm. Through the graph diffusion smoothing technology, the diffusion matrix is ​​converted into a weighted adjacency matrix, which means converting the diffusion probability into the weight of the connection between nodes. For example, if the diffusion probability between two nodes is high, then the corresponding element (weight) in the adjacency matrix will be larger. A sparse matrix refers to a matrix in which most of the elements are zero. In the graph scenario, many nodes may not have a direct connection, so the adjacency matrix is ​​often sparse. Through a specific algorithm or threshold setting, in the process of converting the diffusion matrix to the adjacency matrix, connections with significant weights can be retained, while connections with small or negligible weights can be set to zero, thereby obtaining a sparse adjacency matrix. This can reduce storage and computational costs while highlighting the important connections in the graph. Next, use the adjacency matrix As the probability of polygonal distribution, neighbor nodes are sampled from the one-hop subgraph of the key node to form the adjacency set of the synthetic node. The polygonal distribution here can be understood as a probability distribution based on the graph structure. In this distribution, the neighbor selection probability of a node is determined based on the weight of the adjacency matrix. Each element in the adjacency matrix corresponds to the connection weight between nodes, and these weights can be used as probability values ​​after normalization. For example, for a node v, all its neighbor nodes u 1 ,u 1 ,u 1 ,......,u n The connection weights with node v are w 1 , w 1 ,......,w n , then select neighbor u from node v i The probability P(u i ) can be obtained by To calculate. In this way, the adjacency matrix defines a probability model of polygon distribution for random sampling in the graph. Based on the polygon distribution probability defined previously, neighbor nodes are randomly selected from the one-hop subgraph of the key node. Each time sampling occurs, each neighbor node has a certain probability of being selected according to the probability determined by the adjacency matrix. This process can be repeated multiple times to obtain a sufficient number of neighbor node samples. By sampling neighbor nodes from the one-hop subgraph of the key node, these sampled neighbor nodes constitute the adjacency set of the synthetic node. This adjacency set reflects the local connection structure of the key node in the graph and can be used for subsequent analysis.

[0059] After all the synthesis nodes are obtained, all the synthesis nodes are incorporated into the training set to form a new training set as the basis for subsequent training.

[0060] In some embodiments, Figure 3 As shown, in S130, for each of the multiple training subsets, the following part or all of the content is included.

[0061] S131, calculating the neighbor label distribution of each node in each training subset and the category connection matrix of each category in the total categories corresponding to each training subset;

[0062] S132, calculating the abnormal connection awareness edge ACM value of any node in each training subset in any category of the total categories corresponding to each training subset according to the following formula: in, is the ACM value of node v in category t, is the category y to which node v belongs v The average degree of internal node connections, is a neighbor node of node v that belongs to category y v The ratio of D v，t is the proportion of node v’s neighbor nodes belonging to category t, is category y v The proportion of average connections between and category t;

[0063] S133, calculating the loss function of the sub-model corresponding to each sub-training set according to the following formula: L TAM is the loss function, is the logits of node v, y v is the category of node v, represents the ACM term of node v, α is a hyperparameter;

[0064] S134, optimizing each sub-model based on the loss function of the sub-model corresponding to each sub-training set;

[0065] S135, for each sub-model in the multiple sub-models, after each training using the above steps, the correctly classified nodes in the sub-training set corresponding to each sub-model are removed, and the nodes in the sub-training sets corresponding to other sub-models in the multiple sub-models are added to perform the above training steps again until all nodes in the new training set are correctly classified.

[0066] In S131, it is assumed that Define Neighbor Label Distribution (NLD): For each node v in any training subset, calculate its neighbor label distribution D according to the formula. First, for each training subset, traverse each node in it, and for each node, determine all its neighbor nodes through the edge information of the graph. After that, for each node's neighbor nodes, count the number of occurrences of each category in the neighbor nodes. And divide the number of occurrences of each category by the total number of neighbor nodes to get the neighbor label distribution of the node. For example, if a node has 10 neighbors, 3 of which are labeled "category A", 4 are labeled "category B", and 3 are labeled "category C", then the neighbor label distribution of the node is (0.3, 0.4, 0.3), corresponding to the probabilities of "category A", "category B", and "category C", respectively.

[0067] Similarly, if Define the Class-wise Connectivity Matrix (C): First, clarify the total set of categories involved in each training subset. And for each training subset, create a category connection matrix for each category in the total category. The size of the matrix is ​​usually the number of nodes * the number of nodes. If there is a connection between the nodes and the two node labels of the connection belong to the current category, the element at the corresponding position of the matrix is ​​1, otherwise it is 0.

[0068] In S132, in order to suppress the false alarm caused by abnormal connection nodes, an abnormal connection perception margin (ACM) is defined by calibrating the D of node v. v , (neighborhood label distribution) and category y v of The edge of each category is modified by comparing the deviation between the (connection patterns). By comparing the ratio of the node's own internal connection and the connection with other categories, we can find those nodes that show abnormalities in the inter-category connection. Such anomalies may be of great significance in the fields of network analysis and anomaly detection.

[0069] In S133, by adopting the formula To calculate the loss function of each sub-model, the ACM value is included in the loss function, and the model can better capture abnormalities in node connections. At the same time, considering the ACM value also helps the model to better distinguish between different categories. It can prevent the model from overfitting to the normal connection pattern, thereby improving the model's generalization ability on unknown data.

[0070] After the loss function of each sub-model is obtained, each sub-model can be further optimized using the loss function in S134.

[0071] In S135, during training, the new training set is divided into multiple training subsets, each of which contains a certain number of active nodes and their neighboring nodes. After each sub-model is trained, the correctly classified nodes are removed from the list, and this process is repeated until all nodes are correctly classified or used to train sub-models. The prediction results of all sub-models are integrated, and if a node is correctly classified in at least one sub-model, it is considered to be benign; otherwise, it will be marked as abnormal.

[0072] The main purpose of adopting the multi-model training method is to make full use of the data and continuously adjust the training subsets so that each sub-model can learn more different features and patterns, thereby improving the overall classification accuracy. It avoids the sub-model from overfitting its own training set by dynamically exchanging correctly classified nodes between sub-models, while promoting knowledge sharing between sub-models, which helps to improve the generalization ability and accuracy of the entire model system.

[0073] like Figure 4 As shown, the embodiment of the present application also provides a multi-model unbalanced node classification device. Optionally, as Figure 4 As shown, the multi-model unbalanced node classification device 200 includes part or all of the following contents.

[0074] The acquisition module 210 is used to acquire a graph dataset, where the graph dataset includes a training set and a test set.

[0075] The synthetic node generation module 220 is used to generate synthetic nodes based on the information of each node in the training set, so that the number of nodes in each category in the training set is evenly distributed. The synthetic node generation module 210 is specifically used to:

[0076] Calculate the predicted probability vector and uniform distribution of each node in the training set, and calculate the total variation distance between the predicted probability vector and the uniform distribution of each node, and use it as the uncertainty score of each node,

[0077] The uncertainty score of each node is processed based on the following formula to obtain the node risk value of each node: Among them, U Θ (v) is the uncertainty score of node v, is the number of nodes of category j, is the category predicted by the model The number of nodes, C is the total number of categories in the training set,

[0078] According to the node risk value of each node, a key node is determined from the training set, wherein the key node is a node in the training set that is difficult to be correctly classified,

[0079] Determine the nodes in the training set that are directly connected to the key nodes as auxiliary nodes,

[0080] Generate the characteristics of the synthetic node according to the following formula: X syn =δX anc +(1-δ)X aux , δ is a random value between 0 and 1, X syn is the characteristic of the synthesis node, X anc is the feature of the key node, X aux is the feature of the auxiliary node,

[0081] Sampling from the one-hop subgraph of the key node to generate the edge of the synthetic node, and

[0082] The diffusion matrix S of the training set is converted into a weighted and sparse adjacency matrix using graph diffusion smoothing technology And use the adjacency matrix As the probability of polygon distribution, neighbor nodes are sampled from the one-hop subgraph of the key node to form an adjacency set of the synthetic node.

[0083] The training module 230 is used to add all the generated synthetic nodes to the training set to form a new training set, and divide the new training set into multiple training subsets and input them into multiple sub-models in the graph neural network model for training until the performance of the graph neural network model converges. For each training subset in the multiple training subsets, the training module 230 is specifically used to:

[0084] Calculate the neighbor label distribution of each node in each training subset and the category connection matrix of each category in the total categories corresponding to each training subset,

[0085] The abnormal connection-aware edge ACM value of any node in each training subset in any category of the total categories corresponding to each training subset is calculated according to the following formula: in, is the ACM value of node v in category t, is the category y to which node v belongs v The average degree of internal node connections, is a neighbor node of node v that belongs to category y v The ratio of D v，t is the proportion of node v’s neighbor nodes belonging to category t, is category y v and the average connection ratio between category t,

[0086] The loss function of the sub-model corresponding to each sub-training set is calculated according to the following formula: L TAM is the loss function, is the logits of node v, y v is the category of node v, represents the ACM term of node v, α is a hyperparameter,

[0087] Based on the loss function of the sub-model corresponding to each sub-training set, each sub-model is optimized respectively, and

[0088] For each sub-model in the multiple sub-models, after each training using the above steps, the correctly classified nodes in the sub-training set corresponding to each sub-model are removed, and the nodes in the sub-training sets corresponding to other sub-models in the multiple sub-models are added to perform the above training steps again until all nodes in the new training set are correctly classified;

[0089] The testing module 240 is used to input the test set into the performance-converged graph neural network model for node classification.

[0090] In some embodiments, the training module 230 is specifically used to obtain the uncertainty score of each node based on the following formula: in, is the predicted probability vector of node v The node predicted by the model belongs to the category The uniform distribution of is the predicted probability that node v belongs to category j, is the indicator function, when 1 if the value is 0, otherwise it is 0.

[0091] In some embodiments, the multi-model unbalanced node classification device further includes: a sampling module,

[0092] Used to sample a random value δ from a random distribution.

[0093] In some embodiments, the multi-model unbalanced node classification device also includes: an evaluation module for evaluating whether the graph neural network model has converged based on the accuracy and F1 score of the training results.

[0094] In some embodiments, the graph neural network model includes at least one of a graph convolutional network GCN, a graph attention network GAT, and a graph sampling and aggregation GraphSAGE.

[0095] Based on the same idea, the embodiment of the present application also provides another multi-model unbalanced node classification device. The multi-model unbalanced node classification device includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the program instructions are executed by the processor, the multi-model unbalanced node classification device performs the above-mentioned various method embodiments.

[0096] It should be noted that the detailed contents of the device side embodiment can be found in the method side embodiment, and for the sake of brevity, they will not be repeated here.

[0097] Based on the same idea, an embodiment of the present application also provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned various method embodiments.

[0098] Although the present application has been described with reference to preferred embodiments, various modifications may be made thereto and parts thereof may be replaced with equivalents without departing from the scope of the present application. In particular, the various technical features mentioned in the various embodiments may be combined in any manner as long as there are no structural conflicts. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A multi-model unbalanced node classification method, characterized in that: include: Obtain a graph dataset, where the graph dataset includes a training set and a test set; Based on the information of each node in the training set, generating a synthetic node so that the number of nodes of each category in the training set is evenly distributed, comprising the following steps: Calculating the predicted probability vector and uniform distribution of each node in the training set, and calculating the total variation distance between the predicted probability vector and the uniform distribution of each node, and using it as the uncertainty score of each node; The uncertainty score of each node is processed based on the following formula to obtain the node risk value of each node: Among them, U Θ (v) is the uncertainty score of node v, is the number of nodes of category j in the training set, is the category to which the model predicts node v belongs The number of nodes, C is the total number of categories in the training set; Determining a key node from the training set according to the node risk value of each node, wherein the key node is a node in the training set that is difficult to be correctly classified; Determine the nodes in the training set that are directly connected to the key nodes as auxiliary nodes; Generate the characteristics of the synthetic node according to the following formula: X syn =δX anc +(1-δ)X aux , δ is a random value between 0 and 1, X syn is the characteristic of the synthesis node, X anc is the feature of the key node, X aux is a characteristic of the auxiliary node; Sampling from the one-hop subgraph of the key node to generate the edge of the synthetic node; and The diffusion matrix S of the training set is converted into a weighted and sparse adjacency matrix using graph diffusion smoothing technology And use the adjacency matrix As the probability of polygon distribution, sampling neighbor nodes from the one-hop subgraph of the key node to form an adjacency set of the synthetic node; All generated synthetic nodes are added to the training set to form a new training set, and the new training set is divided into multiple training subsets and respectively input into multiple sub-models in the graph neural network model for training until the performance of the graph neural network model converges. For each training subset in the multiple training subsets, the training process includes the following steps: Calculating the neighbor label distribution of each node in each training subset and the category connection matrix of each category in the total categories corresponding to each training subset; The abnormal connection-aware edge ACM value of each node in each training subset in any category of the total categories corresponding to each training subset is calculated according to the following formula: in, is the ACM value of node v in category t, is the average degree of internal node connections of category yv to which node v belongs, is a neighbor node of node v that belongs to category y v The ratio of D v，t is the proportion of node v’s neighbor nodes belonging to category t, is the category y to which node v belongs v The proportion of average connections between and category t; The loss function of the sub-model corresponding to each sub-training set is calculated according to the following formula: L TAM is the loss function, is the logits of node v, y v is the category of node v, represents the ACM term of node v, α is a hyperparameter; Optimizing each sub-model based on the loss function of the sub-model corresponding to each sub-training set; and For each sub-model in the multiple sub-models, after each training using the above steps, the correctly classified nodes in the sub-training set corresponding to each sub-model are removed, and the nodes in the sub-training sets corresponding to other sub-models in the multiple sub-models are added to perform the above training steps again until all nodes in the new training set are correctly classified; The test set is input into the graph neural network model with converged performance for node classification.

2. The multi-model unbalanced node classification method according to claim 1, characterized in that: The calculating the total variation distance between the predicted probability vector of each node and the uniform distribution and using the total variation distance as the uncertainty score of each node comprises: The uncertainty score of each node is obtained based on the following formula: in, is the predicted probability vector of node v The node predicted by the model belongs to the category The uniform distribution of is the predicted probability that node v belongs to category j, is the indicator function, when 1 if the value is 0, otherwise it is 0.

3. The multi-model unbalanced node classification method according to claim 1, characterized in that: Also includes: A random value δ is sampled from a random distribution.

4. The multi-model unbalanced node classification method according to claim 1, characterized in that: Also includes: Whether the graph neural network model has converged is evaluated based on the accuracy and F1 score of the training results.

5. The multi-model unbalanced node classification method according to claim 1, characterized in that: The graph neural network model includes at least one of a graph convolutional network GCN, a graph attention network GAT, and a graph sampling and aggregation GraphSAGE.

6. A multi-model unbalanced node classification device, characterized in that: include: An acquisition module is used to acquire a graph dataset, where the graph dataset includes a training set and a test set; A synthetic node generation module is used to generate synthetic nodes based on the information of each node in the training set, so that the number of nodes in each category in the training set is evenly distributed. The synthetic node generation module is specifically used to: Calculate the predicted probability vector and uniform distribution of each node in the training set, and calculate the total variation distance between the predicted probability vector and the uniform distribution of each node, and use it as the uncertainty score of each node, The uncertainty score of each node is processed based on the following formula to obtain the node risk value of each node: Among them, U Θ (v) is the uncertainty score of node v, is the number of nodes of category j in the training set, is the category to which the model predicts node v belongs The number of nodes, C is the total number of categories in the training set, According to the node risk value of each node, a key node is determined from the training set, wherein the key node is a node in the training set that is difficult to be correctly classified, Determine the nodes in the training set that are directly connected to the key nodes as auxiliary nodes, Generate the characteristics of the synthetic node according to the following formula: X syn =δX anc +(1-δ)X aux , δ is a random value between 0 and 1, X syn is the characteristic of the synthesis node, X anc is the feature of the key node, X aux is the characteristic of the auxiliary node, Sampling from the one-hop subgraph of the key node to generate the edge of the synthetic node, and The diffusion matrix S of the training set is converted into a weighted and sparse adjacency matrix using graph diffusion smoothing technology And use the adjacency matrix As the probability of polygon distribution, sampling neighbor nodes from the one-hop subgraph of the key node to form an adjacency set of the synthetic node; A training module is used to add all the generated synthetic nodes to the training set to form a new training set, and divide the new training set into multiple training subsets and input them into multiple sub-models in the graph neural network model for training until the performance of the graph neural network model converges. For each training subset in the multiple training subsets, the training module is specifically used to: Calculate the neighbor label distribution of each node in each training subset and the category connection matrix of each category in the total categories corresponding to each training subset, The abnormal connection-aware edge ACM value of each node in each training subset in any category of the total categories corresponding to each training subset is calculated according to the following formula: in, is the ACM value of node v in category t, is the category y to which node v belongs v The average degree of internal node connections, is a neighbor node of node v that belongs to category y v The ratio of D v，t is the proportion of node v’s neighbor nodes belonging to category t, is the category y to which node v belongs v and the average connection ratio between category t, Based on the loss function of the sub-model corresponding to each sub-training set, each sub-model is optimized respectively, and For each sub-model in the multiple sub-models, after each training using the above steps, the correctly classified nodes in the sub-training set corresponding to each sub-model are removed, and the nodes in the sub-training sets corresponding to other sub-models in the multiple sub-models are added to perform the above training steps again until all nodes in the new training set are correctly classified; A testing module is used to input the test set into the performance-converged graph neural network model for node classification.

7. The multi-model unbalanced node classification device according to claim 6, characterized in that: The training module is specifically used for: The uncertainty score of each node is obtained based on the following formula: in, is the predicted probability vector of node v The node predicted by the model belongs to the category The uniform distribution of is the predicted probability that node v belongs to category j, is the indicator function, when 1 if the value is 0, otherwise it is 0.

8. The multi-model unbalanced node classification device according to claim 6, characterized in that: Also includes: The sampling module is used to sample random values ​​δ from random distribution.

9. The multi-model unbalanced node classification device according to claim 6, characterized in that: Also includes: An evaluation module is used to evaluate whether the graph neural network model has converged based on the accuracy and F1 score of the training results.

10. The multi-model unbalanced node classification device according to claim 6, characterized in that: The graph neural network model includes at least one of a graph convolutional network GCN, a graph attention network GAT, and a graph sampling and aggregation GraphSAGE.

Images