Adversarial edge removal device, adversarial edge removal method, and adversarial edge removal program
The adversarial edge removal device and method address the incomplete elimination of adversarial edges in UGCL by selectively removing edges with low feature similarity and high gradient contribution, effectively preventing poisoning attacks and minimizing the loss function in UGCL.
Patent Information
- Application Number
- JP2023198372
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-11-22
- Publication Date
- 2025-06-03
AI Technical Summary
Conventional countermeasures against poisoning attacks on unsupervised graph contrastive learning (UGCL) fail to completely eliminate the influence of adversarial edges, as learning is performed with poisoned graphs still containing these edges.
An adversarial edge removal device and method that optimizes an encoding function for a graph composed of an adjacency matrix and node features using a UGCL model, generates augmented graphs, calculates loss functions and gradients, and selectively removes edges with low feature similarity and high gradient contribution to minimize the loss function.
Effectively removes the influence of adversarial edges from poisoned graphs, preventing poisoning attacks on UGCL without requiring labeled data, and is applicable to any UGCL method by minimizing the loss function.
Smart Images
Figure 2025084457000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a countermeasure method against poisoning attacks on graph neural networks (GNNs).
Background Art
[0002] Conventionally, GNNs that can handle classification tasks for graph data have attracted attention and can be applied to community analysis in social networks, prediction of molecular characteristics in drug discovery, etc. However, in the field of graph learning, it is said that there is a shortage of labeled datasets because the human burden in creating labeled data for specific classification tasks is large, and there is a problem that overfitting and the like are likely to occur. Therefore, unsupervised graph contrastive learning (UGCL) is expected to solve this problem by applying contrastive learning techniques that can learn an encoding function that outputs better representations (numerical vectors) from a large amount of unlabeled data to graph data. However, a poisoning attack that adds adversarial edges to interfere with the learning of UGCL has also been shown, and countermeasures are necessary for the effective use of UGCL.
[0003] As an existing countermeasure against poisoning attacks on UGCL, there is a technique called adversarial learning that is applied to UGCL to acquire resistance to poisoning attacks (see, for example, Non-Patent Document 1). In UGCL, in general, an operation called graph augmentation is performed to create two extended graphs called Views while considering various graph structures, and learning is performed based on them. In the existing method (Non-Patent Document 1), a third graph called an adversarial View is created. By adding a term that calculates the loss between this adversarial View and one of the two Views to the loss function of UGCL and performing learning, it becomes possible to generate an encoder that outputs a better graph representation while acquiring resistance to poisoning attacks.
Prior Art Documents
Non-Patent Documents
[0004]
Non-Patent Document 1
Non-Patent Document 2
Non-Patent Document 3
Summary of the Invention
Problems to be Solved by the Invention
[0005] However, in the conventional countermeasure methods, since learning is performed with a poisoned graph that still contains adversarial edges as input, there is a problem that the influence of the adversarial edges cannot be completely eliminated during learning. For this reason, a method of eliminating the influence of the poisoning attack by deleting adversarial edges from the poisoned graph before learning is desired.
[0006] An object of the present invention is to provide an adversarial edge removal device, an adversarial edge removal method, and an adversarial edge removal program that can appropriately remove unnecessary edges included in a graph that is an input to UGCL.
Means for Solving the Problem
[0007] The adversarial edge removal device according to the present invention optimizes an encoding function for learning a graph composed of an adjacency matrix indicating edges and feature amounts of nodes by a first UGCL model and obtaining an embedded representation of the nodes, and generates a plurality of augmented graphs by performing data augmentation operations on the graph. A representation acquisition unit that obtains an embedded representation of each node of the augmented graph using the encoding function, a gradient calculation unit that calculates a loss function of a second UGCL model based on the embedded representation of each node in each of the plurality of augmented graphs and calculates a gradient of the loss function with respect to the adjacency matrix, and among the edges where the similarity of the feature amounts between the connected nodes in the graph is smaller than a threshold value, an adjacency matrix transformation unit that selects an edge that maximizes the sum of the gradients and inverts the value of the corresponding element of the adjacency matrix, and the graph learning unit, the representation acquisition unit, the gradient calculation unit, and the adjacency matrix transformation unit are repeatedly processed a predetermined number of times, and an output unit that outputs the transformed graph.
[0008] The adversarial edge removal device calculates a loss function of the second UGCL model based on the embedded representation of each node in the graph from which an edge has been removed by the adjacency matrix transformation unit, and includes a graph evaluation unit that holds the adjacency matrix of the graph when the minimum value is obtained. The output unit may output a graph corresponding to the adjacency matrix held by the graph evaluation unit after repeatedly processing the graph learning unit, the representation acquisition unit, the gradient calculation unit, the adjacency matrix transformation unit, and the graph evaluation unit a predetermined number of times.
[0009] The first UGCL model may be the same as the second UGCL model.
[0010] The threshold value may be an average value of the similarity of the feature amounts among all nodes where edges exist.
[0011] The adversarial edge removal method according to the present invention includes a graph learning step of learning a graph composed of an adjacency matrix indicating edges and feature amounts of nodes by a first UGCL model and optimizing an encoding function for obtaining an embedded representation of the nodes; a representation acquisition step of generating a plurality of augmented graphs by performing data augmentation operations on the graph and obtaining the embedded representation of each node of the augmented graphs using the encoding function; a gradient calculation step of calculating a loss function of a second UGCL model based on the embedded representation of each node in each of the plurality of augmented graphs and calculating a gradient of the loss function with respect to the adjacency matrix; an adjacency matrix transformation step of selecting an edge with the largest sum of the gradients among the edges where the similarity of the feature amounts between the connected nodes in the graph is smaller than a threshold value and inverting the value of the element of the corresponding adjacency matrix; and an output step of repeating the processes of the graph learning step, the representation acquisition step, the gradient calculation step, and the adjacency matrix transformation step a predetermined number of times and outputting the transformed graph. A computer executes these steps.
[0012] The adversarial edge removal program according to the present invention is for causing a computer to function as the adversarial edge removal device.
Advantages of the Invention
[0013] According to the present invention, unnecessary edges included in a graph that is an input to UGCL can be appropriately removed.
Brief Description of the Drawings
[0014]
Figure 1
Figure 2
Embodiments for Carrying Out the Invention
[0015] Hereinafter, an example of an embodiment of the present invention will be described. The adversarial edge removal method of this embodiment focuses on the tendency that adversarial edges are added between nodes with dissimilar features when an attacker creates a poisoned graph, and removes the influence of the poison by deleting the edges with low similarity of features between adjacent nodes and that contribute to minimizing the loss function.
[0016] [UGCL method] The UGCL targeted by this embodiment is an algorithm that optimizes (learns) an encoding function for obtaining the embedded representation of a node of interest by comparing the node of interest with similar and dissimilar nodes without using the labels of the nodes in the graph data G = {A, X}. Here, A ∈ {0, 1} N×N is the adjacency matrix, and X ∈ R N×F represents the feature amounts of the nodes, and N and F are the number of nodes in the graph and the dimensionality of the feature amounts of each node, respectively. The embedded representation by the learned encoding function can be used in tasks such as node classification or link prediction.
[0017] In the conventional unsupervised contrastive learning method (Non-Patent Document 2) for graph data assuming node classification and link prediction, first, two data augmentation operations are performed on G, and two augmented graphs are generated. The data augmentation operations shown here refer to modifications of edge connections or masking processes on the features of nodes, etc.
[0018] Next, assuming that the embedded representation E = g{f(A, X)} of G is obtained by the encoding function f(·) and the projection function g(·) used for contrastive learning, for each of the two augmented graphs obtained from the data augmentation operations, the embedded representation E 1 and E 2 are obtained. Here, if the embedded representation of node i (1 ≤ i ≤ N) is e i = E[i, :] ∈ R 1×F then the embedded representations e i 1 and e i 2 of node i in each of the two augmented graphs are obtained.
[0019] After that, learning is performed by optimizing the parameters of the contrastive learning model in the direction of minimizing L using the gradient of the objective function L. [Number] Here, [Number] is defined as, and β(e i 1 , e i 2 ) is the cosine similarity between e i 1 and e i 2 . τ is called the temperature parameter and is a value for controlling the magnitude of the gradient for different nodes.
[0020] [Attack Model] The attacker aims to reduce the accuracy of the embedding representation of the encoding function by having the poisoned graph learned by the UGCL, and as a result, reduce the accuracy of the downstream classification task using the encoding function. The downstream classification tasks are assumed to be node classification for classifying the classes of nodes in the graph and link prediction for predicting whether there is a link between each pair of nodes.
[0021] The poisoned graph used in the poisoning attack is created by selecting effective edges for reducing the accuracy of the embedding representation of the encoding function for a clean graph and modifying it little by little. Specifically, the attacker generates a poisoned graph, for example, by the following procedure (Non-Patent Document 3).
[0022] 1. Establish a surrogate model S of the UGCL method (Non-Patent Document 2) to be attacked. 2. Learn the graph data G with the surrogate model S, perform K data augmentation operations using the encoding function f(·) with optimized parameters and the projection function g(·) used for contrastive learning, and calculate the loss function L of the surrogate model S. 3. Calculate the gradient of \(L\) with respect to the adjacency matrix, select the value of the adjacency matrix where the absolute value of the gradient is maximized, invert one value of the adjacency matrix to maximize \(L\), and obtain a poisoned graph \(PG = \{A', X\}\) containing adversarial edges. 4. Repeat the processes in steps 2 and 3 for a determined number of times for \(PG\) to obtain the final \(PG=\{A', X\}\).
[0023] [Adversarial Edge Removal Method] The adversarial edge removal device 1 of this embodiment removes adversarial edges from the poisoned graph \(PG\) containing the adversarial edges created in this way, and outputs a clean graph \(G\). clean
[0024] FIG. 1 is a diagram showing the functional configuration of the adversarial edge removal device 1 in this embodiment. The adversarial edge removal device 1 is an information processing device (computer) such as a server or a personal computer equipped with various input / output interfaces etc., in addition to a control unit 10 and a storage unit 20.
[0025] The control unit 10 is a part that controls the entire adversarial edge removal device 1, and realizes each function in this embodiment by appropriately reading and executing various programs stored in the storage unit 20. The control unit 10 may be a CPU.
[0026] The storage unit 20 is a storage area for various programs and various data etc. for causing the hardware group to function as the adversarial edge removal device 1, and may be a ROM, a RAM, a flash memory, a hard disk drive (HDD), etc. Specifically, the storage unit 20 stores, in addition to a program (adversarial edge removal program) for causing the control unit 10 to execute each function of this embodiment, parameters of the target UGCL model, graph data, etc.
[0027] The control unit 10 includes a graph learning unit 11, a representation acquisition unit 12, a gradient calculation unit 13, an adjacency matrix conversion unit 14, a graph evaluation unit 15, and an output unit 16. The adversarial edge removal device 1 outputs a clean graph with adversarial edges removed from the graph to be processed by operating these functional units.
[0028] The graph learning unit 11 learns a graph consisting of an adjacency matrix indicating edges and feature amounts of nodes by a first UGCL model, and optimizes an encoding function for obtaining an embedded representation of the nodes.
[0029] The representation acquisition unit 12 generates a plurality of augmented graphs by data augmentation operations on the graph, and obtains an embedded representation of each node of the augmented graphs using the encoding function optimized by the graph learning unit 11. At this time, the representation acquisition unit 12 may generate a plurality of types of graph sets by data augmentation operations on the graph following the normal procedure of contrast learning.
[0030] The gradient calculation unit 13 calculates a loss function of a second UGCL model that is a defense target against attacks based on the embedded representation of each node in each of the plurality of augmented graphs, and calculates a gradient of the loss function with respect to the adjacency matrix.
[0031] The adjacency matrix conversion unit 14 selects an edge among edges where the similarity of feature amounts between connected nodes is less than a threshold value, and for which the total sum of the gradients calculated by the gradient calculation unit 13 is the maximum, and inverts the corresponding element of the adjacency matrix.
[0032] The graph evaluation unit 15 inputs the graph with edges removed by the adjacency matrix conversion unit 14 into the second UGCL model to calculate a loss function, and when the result becomes the minimum value, holds the adjacency matrix at that time.
[0033] The output unit 16 repeatedly performs the processing of the graph learning unit 11, the representation acquisition unit 12, the gradient calculation unit 13, the adjacency matrix conversion unit 14, and the graph evaluation unit 15 a predetermined number of times, and outputs the graph after converting it into an adjacency matrix for which the loss function is the minimum.
[0034] Figure 2 is a flowchart showing the procedure of the adversarial edge removal method in the present embodiment. In step S1, the graph learning unit 11 inputs the poisoned graph PG = {A’, X} including adversarial edges, and learns it with the UGCL model (for example, the method of Non-Patent Document 2) to obtain an encoding function f(·) with optimized parameters. Here, A’ ∈ {0, 1} N×N , X ∈ R N×F where N and F are the number of nodes in the graph and the dimension of the feature amount of each node, respectively.
[0035] In step S2, the representation acquisition unit 12 performs the data augmentation operation K times to create M × K extended graphs (Views) PG m k =(A m k , X m k ). Here, 1 ≤ k ≤ K and 1 ≤ m ≤ M. For example, in the case of the countermeasure method of Non-Patent Document 1, M = 3, and in the case of the UGCL method of Non-Patent Document 2, M = 2.
[0036] In step S3, the representation acquisition unit 12 uses the encoding function f(·) and the projection function g(·) to obtain the embedding representation E m k of PG m k = g{f(A m k , X m k )}.
[0037] In step S4, the gradient calculation unit 13 calculates the loss function L m k ’ of UGCL based on the embedding representation e m,i k of node i in PG. Here, L k ’ is the loss function of an arbitrary UGCL method to be defended. k ’ is the loss function of an arbitrary UGCL method to be defended.
[0038] In step S5, the gradient calculation unit 13 calculates the loss function L mk Adjacency matrix A in m k Regarding L k Gradient ∇ of L’ m k ∈R N×N Are calculated as follows respectively.
Number
[0039] In step S6, the adjacency matrix conversion unit 14 calculates the sum of gradients ∇ total =Σ k,m ∇ m k And selects the set of elements C = {c n |1≦n≦N’} of A’ corresponding to the edges where the sum of gradients is positive and the similarity of feature values between the connected nodes is smaller than a specific value as the edges to be deleted. N’ is the number of edges to be deleted. The conditions for selecting these edges to be deleted can be expressed as follows.
Number
[0040] In step S7, the graph evaluation unit 15 inputs PG’ into the learned UGCL model. If the minimum L’ among the graphs during the processing of this method is calculated as a result of calculating L’, then A’ tmp at that time is held as the optimal adjacency matrix A opt .
[0041] In step S8, the control unit 10 repeats the processes of steps S1 to S7 σ times. That is, if the number of times is less than σ, the process returns to step S1.
[0042] In step S9, the output unit 16 outputs G clean ={A opt , X} as the graph after the adversarial edge deletion process.
[0043] Note that this algorithm can be formulated as the following optimization problem when, for example, M = 2.
Equation
Equation
[0044] According to this embodiment, the adversarial edge removal device 1 removes the edges contributing to the poisoning attack from the poisoned graph used in the poisoning attack on UGCL by gradually removing the edges with low similarity of features between adjacent nodes and contributing to the minimization of the loss function, thereby removing the influence of the poisoned graph. Therefore, the adversarial edge removal device 1 can appropriately remove the adversarial edges contained in the poisoned graph that is input to UGCL. As a result, it is possible to prevent the poisoning attack without using labeled data. In addition, since the loss function of UGCL is an essential element in all existing methods, this embodiment is applicable to any UGCL method, and an effect of preventing the influence of the poisoning attack can be expected.
[0045] The adversarial edge removal device 1 can verify whether it is appropriate to remove the edges searched from only the gradients by saving the adjacency matrix when the loss function of UGCL becomes the minimum. Therefore, among the σ times of repetition of the deletion process, a more appropriate graph can be output. Furthermore, since the adversarial edge removal device 1 removes edges so as to minimize the loss function of UGCL, even when applied to a clean graph that does not include adversarial edges, it can remove the edges that are noise unintentionally included during graph data creation regardless of the attack, and can output a graph that is easy to learn.
[0046] Note that according to the present embodiment, for example, it is possible to prevent poisoning attacks against UGCL, so it is possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Build resilient infrastructure, promote sustainable industrialization, and foster innovation."
[0047] As described above, the embodiments of the present invention have been described. However, the present invention is not limited to the above-described embodiments. Also, the effects described in the above-described embodiments are merely an enumeration of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments.
[0048] The adversarial edge removal method by the adversarial edge removal device 1 is realized by software. When realized by software, the program constituting this software is installed in an information processing device (computer). Also, these programs may be recorded on a removable medium such as a CD-ROM and distributed to users, or may be distributed by being downloaded to the user's computer via a network. Furthermore, these programs may be provided to the user's computer as a web service via a network without being downloaded.
Explanation of Reference Numerals
[0049] 1 Adversarial edge removal device 10 Control Unit 11 Graph Learning Unit 12 Expression Acquisition Unit 13 Gradient Calculation Unit 14 Adjacency Matrix Conversion Unit 15 Graph Evaluation Unit 16 Output Unit 20 Memory Unit
Claims
1. A graph learning unit that optimizes an encoding function for learning a graph composed of an adjacency matrix indicating edges and feature amounts of nodes by a first UGCL (Unsupervised Graph Contrastive Learning) model to obtain an embedded representation of the nodes, A representation acquisition unit that generates a plurality of augmented graphs by performing data augmentation operations on the graph and obtains an embedded representation of each node of the augmented graphs using the encoding function, A gradient calculation unit that calculates a loss function of a second UGCL model based on the embedded representations of the nodes in each of the plurality of augmented graphs and calculates a gradient of the loss function with respect to the adjacency matrix, An adjacency matrix conversion unit that selects an edge for which the sum of the gradients is maximized from among edges for which the similarity of the feature amounts between the nodes connected in the graph is smaller than a threshold value and inverts the value of the element of the corresponding adjacency matrix, An adversarial edge removal device including an output unit that repeatedly performs the processes of the graph learning unit, the representation acquisition unit, the gradient calculation unit, and the adjacency matrix conversion unit a predetermined number of times and outputs the transformed graph.
2. A graph evaluation unit that calculates a loss function of the second UGCL model based on the embedded representation of each node in the graph from which an edge has been removed by the adjacency matrix conversion unit and holds the adjacency matrix of the graph when the loss function becomes a minimum value, The output unit outputs a graph corresponding to the adjacency matrix held by the graph evaluation unit after repeatedly performing the processes of the graph learning unit, the representation acquisition unit, the gradient calculation unit, the adjacency matrix conversion unit, and the graph evaluation unit a predetermined number of times. The adversarial edge removal device according to Claim 1.
3. The adversarial edge removal device according to Claim 1 or Claim 2, wherein the first UGCL model is the same as the second UGCL model.
4. The adversarial edge removal device according to Claim 1 or Claim 2, wherein the threshold value is a value obtained by averaging the similarity of the feature amounts between all nodes where edges exist.
5. A graph learning step of learning a graph composed of an adjacency matrix indicating edges and feature amounts of nodes by a first UGCL (Unsupervised Graph Contrastive Learning) model to optimize an encoding function for obtaining an embedded representation of the nodes, A representation acquisition step of generating a plurality of augmented graphs by performing data augmentation operations on the graph, and obtaining an embedding representation for each node of the augmented graphs using the encoding function; A gradient calculation step of calculating a loss function of the second UGCL model based on the embedding representations of each node in each of the plurality of augmented graphs, and calculating a gradient of the loss function with respect to the adjacency matrix; An adjacency matrix transformation step of selecting an edge with the maximum sum of the gradients among the edges in which the similarity of feature amounts between the connected nodes in the graph is smaller than a threshold value, and inverting the value of the corresponding element of the adjacency matrix; An output step of repeating the processes of the graph learning step, the representation acquisition step, the gradient calculation step, and the adjacency matrix transformation step a predetermined number of times, and outputting the transformed graph. A method for adversarial edge removal executed by a computer.
6. An adversarial edge removal program for causing a computer to function as the adversarial edge removal device according to Claim 1 or Claim 2.