Poison edge removal device, poison edge removal method, and poison edge removal program

Abstract

To provide a poison edge removal device capable of preventing a poisoning attack on unsupervised graph contrastive learning without using labeled data.SOLUTION: A poison edge removal device 1 includes a graph learning unit 11 that learns a graph using an unsupervised graph contrastive learning model and optimizes an encoding function, a representation acquisition unit 12 that generates multiple extended graphs using data extension operations and acquires an embedded representation of each of nodes of the extended graph using the encoding function, a gradient calculation unit 13 that sums up gradients related to an adjacency matrix for an objective function related to a distance from the center of gravity of the embedded representations calculated for the respective multiple extended graphs, an adjacency matrix conversion unit 14 that preferentially selects one of elements of the adjacency matrix with a large absolute value of the gradient and inverts the value so as to maximize the objective function, and an output unit 15 that repeats processing of the graph learning unit 11, the representation acquisition unit 12, the gradient calculation unit 13, and the adjacency matrix conversion unit 14, and outputs a graph after conversion.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This invention relates to a method for countering poisoning attacks against graph neural networks (GNNs). [Background technology]

[0002] Traditionally, graph neural networks (GNNs), capable of handling classification tasks on graph data, have attracted attention and can be applied to community analysis in social networks and prediction of molecular features in drug discovery. However, in the field of graph learning, there is a problem in that labeled datasets are scarce due to the heavy human burden of creating labeled data for specific classification tasks, making it easy for overfitting to occur. Therefore, unsupervised graph-based comparative learning (see, for example, Non-Patent Document 1) is expected to solve this problem by applying comparative learning techniques, which can directly learn from vast amounts of unlabeled data, to graph data. However, poisoning attacks against unsupervised graph-based comparative learning have also been demonstrated (see, for example, Non-Patent Document 2), and countermeasures are necessary.

[0003] As an existing countermeasure against poisoning attacks on graph data, a method has been proposed that utilizes the inherent properties of graph data to convert poisoned graphs into clean graphs (see, for example, Non-Patent Document 3). A clean graph tends to have a low-rank adjacency matrix and is spars because it connects only a limited number of nodes with similar features. On the other hand, when creating toxicity data, there is a tendency to connect nodes with different features in order to reduce the accuracy of downstream classification tasks, thus compromising the similarity of adjacent node features and the low-rank and sparsity of the adjacency matrix inherent in a clean graph. Existing countermeasures utilize this fact to remove toxic edges, thereby generating a clean adjacency matrix. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Zhu, Yanqiao, et al. "Graph contrastive learning with adaptive augmentation." Proceedings of the Web Conference 2021. 2021. [Non-Patent Document 2] Zhang, Sixiao, et al. "Unsupervised graph poisoning attack via contrastive loss back-propagation." WWW '22: Proceedings of the ACM Web Conference 2022. 2022. [Non-Patent Document 3] JIN, Wei, et al. "Graph structure learning for robust graph neural networks." Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. pp. 66-74. 2020. [Summary of the Invention] [Problems to be Solved by the Invention]

[0005] However, existing countermeasure methods assume supervised GNNs and require correct labels to remove the influence of poisoned data. Therefore, it has been difficult to directly apply existing countermeasure methods to unsupervised graph contrastive learning that assumes unlabeled data. Therefore, a method for preventing poisoning attacks against unsupervised graph contrastive learning without using data labels is desired.

[0006] An object of the present invention is to provide a poisoned edge removal device, a poisoned edge removal method, and a poisoned edge removal program that can prevent a poisoning attack against unsupervised graph contrast learning without using labeled data.

Means for Solving the Problems

[0007] The poisoned edge removal device according to the present invention optimizes an encoding function for learning a graph composed of an adjacency matrix indicating edges and feature amounts of nodes by an unsupervised graph contrast learning model and obtaining an embedded representation of the nodes, a graph learning unit, generates a plurality of augmented graphs by performing a data augmentation operation on the graph, and a representation acquisition unit that obtains an embedded representation of each node of the augmented graph using the encoding function, a gradient calculation unit that sums gradients with respect to the adjacency matrix for an objective function related to distances from the centroids of the embedded representations calculated for each of the plurality of augmented graphs, and an adjacency matrix conversion unit that preferentially selects one element of the adjacency matrix having a large absolute value of the gradient and reverses the value so as to maximize the objective function, and an output unit that repeats the processing of the graph learning unit, the representation acquisition unit, the gradient calculation unit, and the adjacency matrix conversion unit a predetermined number of times and outputs the converted graph.

[0008] The representation acquisition unit may generate two types of graph sets by performing a data augmentation operation on the graph, and the gradient calculation unit may sum the gradients of each of the two types of graph sets.

[0009] The poison edge removal method according to the present invention includes a graph learning step in which a graph consisting of an adjacency matrix representing edges and node features is learned by an unsupervised graph-contrastive learning model and an encoding function for obtaining node embedding representations is optimized; a representation acquisition step in which a plurality of augmented graphs are generated by performing data augmentation operations on the graph and embedding representations of each node in the augmented graphs are obtained using the encoding function; a gradient calculation step in which the gradients with respect to the adjacency matrix are summed for an objective function relating to the distance from the centroid of the embedding representations calculated in each of the plurality of augmented graphs; an adjacency matrix transformation step in which one element of the adjacency matrix with a large absolute value of the gradient is selected preferentially and its value is inverted to maximize the objective function; and an output step in which the graph learning step, the representation acquisition step, the gradient calculation step, and the adjacency matrix transformation step are repeated a predetermined number of times and the transformed graph is output.

[0010] The poison edge removal program according to the present invention is for causing a computer to function as the poison edge removal device. [Effects of the Invention]

[0011] According to the present invention, poisoning attacks against unsupervised graph-controlled learning can be prevented without using labeled data. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows the functional configuration of the poison edge removal device in the embodiment. [Figure 2] This flowchart shows the procedure for removing the poison edge in the embodiment. [Modes for carrying out the invention]

[0013] An example of an embodiment of the present invention will be described below. In poisoning attacks against unsupervised graph-controlled learning, poisoned edges are added between nodes that are actually only loosely related. As a result, while embedding representations should be distributed in unsupervised graph-controlled learning, the embedding representations of nodes affected by poisoned edges are distributed near the centroid, leading to a decrease in accuracy for classification tasks and other similar tasks. The poison edge removal device of this embodiment focuses on this characteristic and removes the influence of poison edges by modifying the adjacency matrix of the input graph to maximize the distance between the embedding representation of each node and its centroid, thereby outputting a clean graph.

[0014] [Unsupervised graph-based learning] The unsupervised graph-symmetric learning targeted by this embodiment refers to graph data G={A,X}(A∈{0,1} N×N The adjacency matrix is ​​X∈R. N×F This algorithm optimizes (learns) an encoding function to obtain an embedding representation of a node of interest by comparing it with similar and different nodes, without using the node's label (where represents the node's feature). The embedded representation using the learned encoding function can be used for tasks such as node classification or link prediction.

[0015] In existing unsupervised controlled learning methods for graph data intended for node classification and link prediction, two data augmentation operations are first performed on G, generating two augmented graphs. These data augmentation operations refer to modifications of edge connections or masking of node features, etc.

[0016] Next, assuming that the embedding representation e = g{f(A,X)} is obtained using the encoding function f(·) and the projection function g(·) used for comparative learning, then the embedding representation e in the two augmented graphs obtained from the data augmentation operation 1 ={e i 1 |1≦i≦N} and e 2 ={e i 2 We obtain the result |1≦i≦N}.

[0017] After that, learning is performed by optimizing the parameters of the contrast learning model in the direction of minimizing L using the gradient of the objective function L.

Number

Number

[0018] [Attack Model] The attacker aims to reduce the accuracy of the embedding representation of the encoding function by having the poisoned graph learned in unsupervised graph contrast learning, and as a result, reduce the accuracy of the downstream classification task using the encoding function. The downstream classification tasks are assumed to be node classification for classifying the classes of nodes in the graph and link prediction for predicting whether there is a link between each pair of nodes.

[0019] The poisoned graph is created by selecting effective edges and modifying them little by little with respect to the clean graph in order to reduce the accuracy of the embedding representation of the encoding function. Specifically, the attacker generates the poisoned graph, for example, by the following procedure.

[0020] 1. Establish a surrogate model S of the unsupervised graph contrast learning method to be attacked. 2. Learn the graph data G with the surrogate model S, perform K data augmentation operations using the encoding function f(·) whose parameters are optimized and the projection function g(·) used for contrast learning, and calculate the loss function L of the surrogate model S. 3. Calculate the gradient of L with respect to the adjacency matrix, select the adjacency matrix value that maximizes the absolute value of the gradient, and invert one value of the adjacency matrix to maximize L, thereby obtaining a poison graph PG={A',X} that includes poison edges. 4. By repeating steps 2 and 3 on PG a predetermined number of times, the final PG = {A', X} is obtained.

[0021] [How to remove poison edge] The poison edge removal device 1 of this embodiment removes poison edges from the poison graph PG containing the poison edges created in this way, resulting in a clean graph G clean Outputs.

[0022] Figure 1 shows the functional configuration of the poison edge removal device 1 in this embodiment. The poison edge removal device 1 is an information processing device (computer) such as a server or personal computer, which is equipped with a control unit 10 and a storage unit 20, as well as various input / output interfaces.

[0023] The control unit 10 controls the entire poison edge removal device 1 and realizes each function in this embodiment by appropriately reading and executing various programs stored in the memory unit 20. The control unit 10 may be a CPU.

[0024] The memory unit 20 is a storage area for various programs and data that enable the hardware group to function as a poison edge removal device 1, and may be ROM, RAM, flash memory, or a hard disk drive (HDD). Specifically, the memory unit 20 stores a program (poison edge removal program) for causing the control unit 10 to execute each function of this embodiment, as well as parameters of the target unsupervised controlled learning model, graph data, and the like.

[0025] The control unit 10 comprises a graph learning unit 11, a representation acquisition unit 12, a gradient calculation unit 13, an adjacent matrix transformation unit 14, and an output unit 15. The poison edge removal device 1 outputs a clean graph from which poison edges have been removed by operating these functional units.

[0026] The graph learning unit 11 learns a graph consisting of an adjacency matrix representing edges and node features using an unsupervised graph-contrastive learning model, and optimizes an encoding function to obtain an embedding representation of the nodes.

[0027] The representation acquisition unit 12 generates multiple augmented graphs by performing data augmentation operations on the graph, and acquires the embedding representation of each node in the augmented graph using the encoding function optimized by the graph learning unit 11. In this case, the representation acquisition unit 12 may generate two sets of graphs by performing data augmentation operations on the graphs, following the usual procedure for comparative learning.

[0028] The gradient calculation unit 13 sums the gradients with respect to the adjacency matrix for the objective function relating to the distance from the centroid of the embedding representation calculated for each of the multiple extended graphs. If two types of graph sets are generated in the representation acquisition unit 12, the gradient calculation unit 13 sums the gradients of each of the two types of graph sets.

[0029] The adjacency matrix transformation unit 14 prioritizes selecting the element of the adjacency matrix with the largest absolute value of the gradient calculated by the gradient calculation unit 13, and inverts its value to maximize the objective function.

[0030] The output unit 15 repeats the processing of the graph learning unit 11, the representation acquisition unit 12, the gradient calculation unit 13, and the adjacency matrix transformation unit 14 a predetermined number of times, and outputs the transformed graph.

[0031] Figure 2 is a flowchart showing the procedure for removing the poison edge in this embodiment. In step S1, the graph learning unit 11 takes a poison graph PG={A',X} containing poison edges as input, trains an unsupervised graph-controlled learning model, and obtains an encoded function f(·) with optimized parameters. Here, A∈{0,1} N×N X∈R N×F Here, N and F are the number of nodes in the graph and the number of dimensions of the feature vectors for each node, respectively.

[0032] In step S2, the representation acquisition unit 12 performs two types of data augmentation operations M times each, and two types of graph sets PG1 k =(A1 k X1 k ) and PG2 k =(A2 k ,X2 k Create a ) where 1 ≤ k ≤ M.

[0033] In step S3, the representation acquisition unit 12 uses the encoding function f(·) and the projection function g(·) to perform PG1 k and PG2 k Node embedding representation e1 k =g{f(A1 k X1 k )} and e2 k =g{f(A2 k ,X2 k )} are obtained respectively.

[0034] In step S4, the gradient calculation unit 13 calculates e1 k ={e 1,i k |1≦i≦N} and e2 k ={e 2,i k The centroid g1 in |1≦i≦N} k and g2 k Calculate each of them.

[0035] In step S5, the gradient calculation unit 13 determines the objective function to be:

number

[0036] In step S6, the gradient calculation unit 13 calculates PG1 k and PG2 k Adjacent matrix A1 k and A2 k The gradient Δ1 of L' with respect to k , Δ2 k These are calculated as follows:

number

[0037] In step S7, the adjacency matrix transformation unit 14 calculates the overall gradient Δ = Σ k=1 M (Δ1 k +Δ1 k The following steps are performed: calculate the value of L', select one element of A' corresponding to the edge with the largest absolute value of the gradient, invert the value (0 or 1) to maximize L', generate a new adjacency matrix A', and obtain a new graph PG'={A',X}.

[0038] In step S8, the control unit 10 repeats the process from steps S1 to S7 σ times. That is, if it does not repeat σ times, the process returns to step S1.

[0039] In step S9, the output unit 15 outputs graph G after the poison edge removal process. clean Outputs.

[0040] Furthermore, this algorithm can be formulated as the following optimization problem.

number

[0041] According to this embodiment, the poison edge removal device 1 removes edges contributing to a poisoning attack from a poison graph used in a poisoning attack against unsupervised graph-controlled learning, thereby eliminating the influence of the poison graph. The attacker modifies the adjacency matrix in small increments when creating the poison graph, and noticing that the embedding representations of nodes affected by poison edges tend to be distributed near the centroid of the overall embedding representation, the poison edge removal device 1 modifies the adjacency matrix in small increments so that the embedding representations of the nodes move away from the centroid. Therefore, the poison edge removal device 1 performs adjacency matrix modification processing that utilizes the characteristics of the distribution of node embedding representations in the poison graph, making it possible to prevent poisoning attacks against unsupervised graph-controlled learning without using labeled data.

[0042] Even if a learning method resistant to attacks is proposed, such a method would propose a new objective function that assumes the presence of toxic data. Therefore, attackers could use the countermeasured learning method to launch new attacks. In contrast, the poison edge removal device 1 removes poison data from the graph used for learning as a preprocessing step before learning, thus providing resistance to attacks that utilize learning methods.

[0043] Furthermore, the poison edge removal device 1 can easily generate a clean graph by reusing existing procedures by generating two types of graph sets using the data augmentation process used in controlled learning.

[0044] Furthermore, since the node embedding representation using unsupervised graph-controlled learning is a common element across all controlled learning methods, it is expected to be effective in preventing the effects of poisoning attacks, regardless of which controlled learning method the attacker uses when creating poisoned data.

[0045] Furthermore, this embodiment makes it possible to prevent poisoning attacks against unsupervised graph-controlled learning without using labeled data, thus contributing to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote sustainable industrialization and foster innovation."

[0046] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. Furthermore, the effects described in the embodiments described above are merely a list of the most preferred effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments.

[0047] The poison edge removal method by the poison edge removal device 1 is implemented by software. When implemented by software, the programs constituting this software are installed on an information processing device (computer). These programs may be recorded on removable media such as a CD-ROM and distributed to users, or they may be distributed by downloading them to the user's computer via a network. Furthermore, these programs may be provided to the user's computer as a web service via a network without being downloaded. [Explanation of Symbols]

[0048] 1. Poison Edge Removal Device 10 Control Unit 11. Graph Learning Section 12 Expression acquisition part 13. Gradient Calculation Unit 14 Adjacent Matrix Transformation Unit 15 Output section 20 Memory section

Claims

1. A graph learning unit learns a graph consisting of an adjacency matrix representing edges and node features using an unsupervised graph-symmetric learning model, and optimizes an encoding function to obtain an embedding representation of the nodes. A representation acquisition unit that generates multiple augmented graphs by performing data augmentation operations on the aforementioned graph, and obtains the embedded representation of each node of the augmented graph using the encoding function, A gradient calculation unit that sums the gradients with respect to the adjacency matrix for the objective function relating to the distance from the centroid of the embedding representation calculated in each of the plurality of extended graphs, An adjacency matrix transformation unit selects one element of the adjacency matrix prioritizing the element with the largest absolute value of the gradient, and inverts its value to maximize the objective function. A poison edge removal device comprising: an output unit that repeats the processing of the graph learning unit, the representation acquisition unit, the gradient calculation unit, and the adjacency matrix transformation unit a predetermined number of times and outputs the transformed graph.

2. The representation acquisition unit generates two types of graph sets by performing data augmentation operations on the graph, The poison edge removal device according to claim 1, wherein the gradient calculation unit sums the gradients of each of the two types of graph sets.

3. A graph learning step involves learning a graph consisting of an adjacency matrix representing edges and node features using an unsupervised graph-symmetric learning model, and optimizing an encoding function to obtain an embedding representation of the nodes. A representation acquisition step involves generating multiple augmented graphs by performing data augmentation operations on the aforementioned graph, and obtaining the embedded representation of each node of the augmented graph using the encoding function, A gradient calculation step of summing the gradients with respect to the adjacency matrix for the objective function relating to the distance from the centroid of the embedding representation calculated for each of the plurality of extended graphs, An adjacency matrix transformation step involves selecting one element of the adjacency matrix that has a large absolute value of the gradient, prioritizing that element, and inverting its value to maximize the objective function. A poison edge removal method in which a computer performs the following steps: repeating the graph learning step, the representation acquisition step, the gradient calculation step, and the adjacency matrix transformation step a predetermined number of times, and outputting the transformed graph.

4. A poison edge removal program for causing a computer to function as a poison edge removal device according to claim 1 or claim 2.

Images