Data expansion generation device, data expansion generation method, and data expansion generation program

Abstract

To provide a device which can generate data extensions robust against backdoor attacks.SOLUTION: A data extension generator 1 comprises: an input unit 11 for receiving input of a first graph; a side spread unit 12 for acquiring a second graph in which the first graph is side-spread; a global expression acquisition unit 13 for acquiring a global expression of the first and the second graphs; an extension processing unit 14 for acquiring the data extension of the first and the second graphs; a local expression acquisition unit 15 for acquiring the local and global expressions of the data extensions of the first and the second graphs; a similarity calculation unit 16 for calculating a similarity between the first and the second graphs; a loss calculation unit 17 for calculating losses based on the similarity; a learning processing unit 18 for calculating the gradient of losses and updating a neural network; and an output unit 19 for outputting the local and global expressions of the data extensions of the first and the second graphs to a new graph.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This invention relates to a method for countering backdoor attacks against graph neural networks (GNNs). [Background technology]

[0002] While GNNs are an important technology for classifying graph-structured data with high accuracy, it has recently been pointed out that backdoor attacks against GNNs are possible. A backdoor attack against a GNN involves an attacker mixing graph data with subgraphs called triggers (poisoned data) into the training data, and then training the GNN with this poisoned data. The goal is to cause the GNN to misclassify only the data with triggers into the attacker's target class during inference.

[0003] The main backdoor attacks against GNNs include a label-flipping attack using fixed triggers, as shown in Non-Patent Document 1 (an attack that changes the label to a target class when poison data is mixed into the training data), and a label-flipping attack using adaptive triggers, as shown in Non-Patent Document 2 (where the trigger is optimized for each graph).

[0004] While few countermeasures against backdoor attacks on GNNs are known, Non-Patent Document 3 proposes an algorithm that identifies poison graphs and removes triggers based on the fidelity and infidelity metrics of Explainable AI (XAI) in response to backdoor attacks based on fixed triggers.

[0005] Furthermore, Graph Contrastive Learning (GCL) is a graph learning method that aims to improve the usefulness of augmented graphs (data augmentation) and the performance of the model by augmenting the graph. Examples of models achieving higher classification accuracy when trained on data augmentation compared to training on ordinary graphs have been reported in Non-Patent Document 4 (MVGRL) and Non-Patent Document 5 (LG2AR), among others. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Zaixi Zhang et al. "Backdoor Attacks to Graph Neural Networks". In: Proceedings of the 26th ACM Symposium on Access Control Models and Technologies. SACMAT '21. 2021, pp. 15-26. [Non-Patent Document 2] Zhaohan Xi et al. "Graph Backdoor". In: 30th USENIX Security Symposium (USENIX Security 21). 2021, pp. 1523-1540. [Non-Patent Document 3] Bingchen Jiang and Zhao Li. "Defending Against Backdoor Attack on Graph Neural Network by Explainability". 2022. [Non-Patent Document 4] Kaveh Hassani and Amir Hosein Khasahmadi. "Contrastive Multi-View Representation Learning on Graphs". In: Proceedings of the 37th International Conference on Machine Learning. ICML '20. 2020. [Non-Patent Document 5] Kaveh Hassani and Amir Hosein Khasahmadi. "Learning graph augmentations to learn graph representations". In: arXiv preprint arXiv:2201.09830 (2022). [Overview of the project] [Problems that the invention aims to solve]

[0007] Conventional methods (Non-Patent Literature 1) are countermeasures against backdoor attacks using fixed triggers, and their effectiveness in defending against more sophisticated backdoor attacks using adaptive triggers has not been demonstrated. Furthermore, because subgraphs identified as triggers are removed from graphs identified as poisonous, it has been experimentally shown that misidentification and erroneous removal can reduce the classification accuracy of clean graphs that do not contain poison.

[0008] The present invention aims to provide a data augmentation generation device, a data augmentation generation method, and a data augmentation generation program that can generate data augments that are robust against backdoor attacks. [Means for solving the problem]

[0009] The data augmentation generation device according to the present invention includes: an input unit that receives input of a set of first graphs; an edge diffusion unit that obtains a set of second graphs obtained by performing edge diffusion on the adjacency matrix of the first graphs; a global representation acquisition unit that inputs the first graphs and the second graphs to a first encoder composed of a neural network and obtains the global representation of the first graphs and the global representation of the second graphs; an augmentation processing unit that inputs the first graphs and their global representations, and the second graphs and their global representations, to an augmenter composed of a neural network and obtains data augmentation of the first graphs and data augmentation of the second graphs; and inputs the data augmentation of the first graphs and the data augmentation of the second graphs to a second encoder composed of a neural network and obtains local and global representations of the data augmentation of the first graphs and local and global representations of the data augmentation of the second graphs. The system includes: a local representation acquisition unit that acquires a global representation; a similarity calculation unit that calculates the similarity between the local representation of the data augmentation of the first graph and the global representation of the data augmentation of the second graph, and the similarity between the local representation of the data augmentation of the second graph and the global representation of the data augmentation of the first graph; a loss calculation unit that calculates a first loss designed to decrease as the similarity between representations based on the same graph increases and as the similarity between representations based on different graphs decreases; a learning processing unit that calculates the gradient of the first loss and updates the parameters of the neural network in the first encoder, the augmenter, and the second encoder; and an output unit that outputs the local and global representations of the data augmentation of the first graph and the local and global representations of the data augmentation of the second graph, when a new graph is made the first graph using the learned first encoder, the augmenter, and the second encoder.

[0010] If the class information of the first graph is available, the loss calculation unit may further calculate a second loss that is designed to decrease as the similarity between representations of the same class increases and as the similarity between representations of different classes decreases. The learning processing unit may then calculate the gradient of the second loss and update the parameters of the neural network in the first encoder, the augmenter, and the second encoder.

[0011] The data augmentation generation method according to the present invention includes an input step of receiving a set of first graphs as input; an edge diffusion step of obtaining a set of second graphs obtained by performing edge diffusion on the adjacency matrix of the first graphs; a global representation acquisition step of inputting the first graphs and the second graphs to a first encoder composed of a neural network and obtaining the global representation of the first graphs and the global representation of the second graphs; an augmentation processing step of inputting the first graphs and their global representations, and the second graphs and their global representations, to an augmenter composed of a neural network and obtaining the data augmentation of the first graphs and the data augmentation of the second graphs; and inputting the data augmentation of the first graphs and the data augmentation of the second graphs to a second encoder composed of a neural network and obtaining the local representation and global representation of the data augmentation of the first graphs and the local representation and global representation of the data augmentation of the second graphs. The computer performs the following steps: a local representation acquisition step to acquire a representation; a similarity calculation step to calculate the similarity between the local representation of the data augmentation of the first graph and the global representation of the data augmentation of the second graph, and the similarity between the local representation of the data augmentation of the second graph and the global representation of the data augmentation of the first graph; a loss calculation step to calculate a first loss designed such that the value decreases as the similarity between representations based on the same graph increases and as the similarity between representations based on different graphs decreases; a learning process step to calculate the gradient of the first loss and update the parameters of the neural network in the first encoder, the augmenter, and the second encoder; and an output step to output the local and global representations of the data augmentation of the first graph and the local and global representations of the data augmentation of the second graph, when a new graph is designated as the first graph, using the trained first encoder, the augmenter, and the second encoder.

[0012] The data augmentation generation program according to the present invention is for causing a computer to function as the data augmentation generation device. [Effects of the Invention]

[0013] According to the present invention, it is possible to generate data augmentations that are robust against backdoor attacks. [Brief explanation of the drawing]

[0014] [Figure 1] This figure shows the functional configuration of the data augmentation and generation device in the embodiment. [Figure 2] This figure shows the learning algorithm for the data augmentation generation method in the embodiment. [Figure 3] This is a schematic diagram illustrating the procedure for the data augmentation generation method in the embodiment. [Modes for carrying out the invention]

[0015] An example of an embodiment of the present invention will be described below. The data augmentation generation method of this embodiment is a method for generating data augmentation and its latent representation that improve the classification accuracy of a model by using GCL, and that is resistant to backdoor attacks, which are a type of attack method against machine learning systems using GNNs. While using existing GCLs such as MVGRL (Non-Patent Literature 4) or LG2AR (Non-Patent Literature 5) directly as countermeasures against backdoor attacks cannot be expected to reduce the success rate of attacks, in this embodiment, unique modifications are made to the GCL learning process so that it is difficult for the GCL to learn triggers.

[0016] The novel graph-based learning algorithm in this embodiment, LG4PD (Learning Graph Augmentations for Poisoned Graph Dataset), is a learning algorithm that includes a neural network that takes a graph G as input and outputs a representation H based on data augmentation of G.

[0017] The data augmentation generator first trains the LG4PD by feeding it the entire training data, including the poisonous data. Using the trained LG4PD, it transforms the entire training data, including the poisonous data, into a data augmentation. This data augmentation or its latent representation, along with the labels, can be fed to a downstream classification model such as an SVM (Support Vector Machine) to train the downstream classification model. During inference by the downstream model, the input graph is transformed into the data augmentation or its latent representation using the trained LG4PD, and then fed to the downstream classification model.

[0018] Figure 1 shows the functional configuration of the data expansion generation device 1 that performs LG4PD in this embodiment. The data expansion and generation device 1 is an information processing device (computer) such as a server or personal computer, which is equipped with a control unit 10 and a storage unit 20, as well as various input / output interfaces.

[0019] The control unit 10 is the part that controls the entire data expansion generation device 1, and realizes each function in this embodiment by appropriately reading and executing various programs stored in the storage unit 20. The control unit 10 may be a CPU.

[0020] The storage unit 20 is a storage area for various programs and various data that enable the hardware group to function as a data expansion generation device 1, and may be ROM, RAM, flash memory, or a hard disk drive (HDD). Specifically, the memory unit 20 stores a program (data augmentation generation program) for causing the control unit 10 to execute each of the functions of this embodiment, as well as neural network parameters, graph data, and the like.

[0021] The control unit 10 comprises an input unit 11, an edge diffusion unit 12, a global representation acquisition unit 13, an extension processing unit 14, a local representation acquisition unit 15, a similarity calculation unit 16, a loss calculation unit 17, a learning processing unit 18, and an output unit 19. The data augmentation generator 1 outputs a latent representation of data augmentation for the input graph by operating these functional units.

[0022] Figure 2 shows the learning algorithm (LG4PD) of the data augmentation generation method in this embodiment. The data augmentation generator 1, using this learning algorithm, takes a set of graphs G as input and generates a latent representation of the graphs.

number

[0023] Line 1: The control unit 10 samples N graphs using the input unit 11. Here,

number

number

[0024] Second line: The control unit 10 controls each graph G i Execute lines 3 through 9 for the result. Third line: The control unit 10, through the edge diffusion unit 12, generates graph G i Adjacent matrix A i Edge diffusion is performed on the resulting adjacency matrix ^A. i This is obtained. This process is the same as MVGRL (Non-Patent Document 4).

[0025] Line 4: The control unit 10, using the global representation acquisition unit 13, retrieves the original graph G i g is an encoder for ω Applying global expression

number

[0026] Line 5: The control unit 10 causes g i to act on the edge-diffused graph ^G ω to obtain the global representation

Number

[0027] Line 6: The control unit 10 inputs the original graph G φ and the global representation z i into the augmenter τ 1 i (GNN) and outputs the data augmentation G i of G 1 i . τ φ consists of a neural network that outputs the probability distribution of each node or edge of G 1 i from z i and a sampler that samples the nodes and edges with high probabilities from the output of the neural network. This method follows LG2AR (Non-Patent Document 5), but while the input of LG2AR is the global representation and the local representation, this method uses only the global representation.

[0028] Line 7: The control unit 又 10 inputs the edge-diffused graph ^G φ and the global representation z i into the augmenter τ 2 i (GNN) and outputs the data augmentation G i of ^G 2 i .

[0029] Line 8: The control unit 10 performs data expansion G using the local representation acquisition unit 15. 1 i to another encoder g θ Enter it into G 1 i Local expression

number

number

[0030] Line 9: The control unit 10 performs data expansion G using the local representation acquisition unit 15. 2 i to another encoder g θ Enter it into G 2 i Local expression

number

number

[0031] Line 10: The control unit 10 controls each set of graphs (G i ,G j Execute lines 11 through 12 for ). Line 11: The control unit 10 calculates the similarity between the first local representation and the second global representation using the similarity calculation unit 16. Local representation H V and global expression h g Similarity I(H) between V ,h g ) is defined as follows:

number

[0032] Line 12: The control unit 10 calculates the similarity between the second local representation and the first global representation using the similarity calculation unit 16.

[0033] Line 13: The control unit 10 calculates the loss L defined by the following equation using SSCL (Self-Supervised Contrastive Loss) based on the similarity of all pairs (i,j) obtained in line 11, using the loss calculation unit 17. 1 self We seek.

number

[0034] Line 14: The control unit 10 uses the SSCL to obtain the loss L from the similarities for all pairs (i, j) obtained in line 12 by the loss calculation unit 17. 2 self to obtain.

[0035] Line 15: When the class information y i of each graph G i (label) is available, the control unit 10 uses the SCL (Supervised Contrastive Loss) to obtain the loss L defined by the following equation from the similarities for all pairs (i, j) obtained in line 11 by the loss calculation unit 17. 1 sup to obtain.

Equation

[0036] Line 16: When the class information y i of each graph G i (label) is available, the control unit 10 uses the SCL to obtain the loss L 2 sup from the similarities for all pairs (i, j) obtained in line 12 by the loss calculation unit 17.

[0037] Line 17: The control unit 10 calculates the gradients of L 1 self and L 2 self by the learning processing unit 18 and updates the parameters of each neural network. Line 18: The control unit 10 calculates the gradients of L 1 sup and L2 sup Calculate the gradient and update the parameters of each neural network.

[0038] FIG. 3 is a schematic diagram showing the procedure of the data augmentation generation method in the present embodiment. The data augmentation generation device 1 has a graph G with the number of nodes n i For, a graph ^G with edges diffused i When generating, apply the augmenter τ φ To generate data augmentations G 1 i And G 2 i Generate. At this time, the input to the augmenter τ φ In addition to the graph (G i Or ^G i ), the global representation (z ω ) of the graph compressed to m (1 ≤ m < n) dimensions by the encoder g 1 i Or z 2 i ).

[0039] Next, the data augmentation generation device 1 compresses the obtained data augmentations G 1 i And G 2 i By the encoder g θ To generate an n-dimensional local representation and an m-dimensional global representation. The data augmentation generation device 1 performs contrast learning based on the similarity of comparing the local and global representations of these two sets of latent representations, and learns the neural networks of g ω , τ φ , g θ . In the downstream classification model, learning and inference are performed with the data augmentation or its latent representation as the input.

[0040] According to the present embodiment, the data augmentation generation device 1 has an augmenter τ φWhen acquiring data augmentation, the global representation compressed by the GNN is used as input along with the graph data. Local representations include information for each node, including the node information of triggers. Global representations aggregate information for each node, and since triggers are usually considered noise from the perspective of the entire graph, aggregation makes it easier for trigger information to get buried among other information. Also, because triggers appear rarely from the perspective of the entire dataset, the probability of the existence of nodes and edges corresponding to triggers is small, and the augmenter τ φ The probability of being sampled is also kept low.

[0041] Therefore, the data augmentation generator 1 can generate data augmentations and their latent representations that are robust against backdoor attacks, and furthermore, by using comparative learning, it is possible to achieve both a reduction in the attack success rate and an improvement in classification accuracy. In this case, the data augmentation generator 1 can appropriately adjust the dimensional m of the global representation to reduce the attack success rate while maintaining practical classification accuracy, thereby appropriately setting a trade-off between reducing the attack success rate and improving classification accuracy.

[0042] The data augmentation generation device 1 is an augmenter τ φ Because local representation is not used when acquiring data augmentation, there was a possibility that the classification accuracy would be lower compared to the existing LG2AR method. To prevent this, the data augmentation generator 1 can improve the usefulness of data augmentation by using SCL, which has the effect of increasing the similarity between the same class and decreasing the similarity between different classes based on label information. Furthermore, the poison data inserted through a label flip attack, which was originally data from a different class, can be made to resemble data from the labeled class by using SCL. As a result, the improved classification accuracy allows for a smaller dimension m in the global representation, thereby further reducing the success rate of attacks.

[0043] Furthermore, this embodiment enables the implementation of general countermeasures against backdoor attacks on GNNs, for example, thereby contributing to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote sustainable industrialization and foster innovation."

[0044] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. Furthermore, the effects described in the embodiments described above are merely a list of the most preferred effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments.

[0045] The data expansion generation method by the data expansion generation device 1 is implemented by software. When implemented by software, the programs constituting this software are installed on an information processing device (computer). These programs may be distributed to users by being recorded on removable media such as a CD-ROM, or by being downloaded to the user's computer via a network. Furthermore, these programs may be provided to the user's computer as a web service via a network without being downloaded. [Explanation of Symbols]

[0046] 1. Data expansion and generation device 10 Control Unit 11 Input section 12-sided diffusion section 13 Global Expression Acquisition Department 14. Extended Processing Unit 15. Local representation acquisition unit 16 Similarity calculation part 17 Loss calculation section 18 Learning Processing Unit 19 Output section 20 Memory section

Claims

1. An input unit that accepts input of a set of first graphs, An edge diffusion unit obtains a set of second graphs obtained by performing edge diffusion on the adjacency matrix of the first graph, A global representation acquisition unit inputs the first graph and the second graph into a first encoder, each composed of a neural network, and acquires the global representation of the first graph and the global representation of the second graph. An augmentation processing unit inputs the first graph and its global representation, and the second graph and its global representation, to an augmenter composed of a neural network, respectively, and obtains data augmentation for the first graph and data augmentation for the second graph. A local representation acquisition unit inputs the data augmentation of the first graph and the data augmentation of the second graph to a second encoder composed of a neural network, and acquires the local and global representations of the data augmentation of the first graph and the local and global representations of the data augmentation of the second graph. A similarity calculation unit calculates the similarity between the local representation of the data extension of the first graph and the global representation of the data extension of the second graph, and the similarity between the local representation of the data extension of the second graph and the global representation of the data extension of the first graph. A loss calculation unit calculates a first loss that is designed to decrease in value as the similarity between representations based on the same graph increases, and as the similarity between representations based on different graphs decreases. A learning processing unit calculates the gradient of the first loss and updates the parameters of the neural network in the first encoder, the augmenter, and the second encoder, A data augmentation generation device comprising: an output unit that outputs local and global representations of the data augmentation of the first graph, and local and global representations of the data augmentation of the second graph, when a new graph is made the first graph using the first encoder, the augmenter, and the second encoder that have been trained; and an output unit that outputs local and global representations of the data augmentation of the second graph.

2. If the class information of the first graph is available, The loss calculation unit further calculates a second loss, which is designed to decrease as the similarity between representations of the same class increases and as the similarity between representations of different classes decreases. The data augmentation generation device according to claim 1, wherein the learning processing unit calculates the gradient of the second loss and updates the parameters of the neural network in the first encoder, the augmenter, and the second encoder.

3. An input step that accepts input of a first set of graphs, An edge diffusion step to obtain a set of second graphs obtained by performing edge diffusion on the adjacency matrix of the first graph, A global representation acquisition step involves inputting the first graph and the second graph into a first encoder, each composed of a neural network, to obtain the global representation of the first graph and the global representation of the second graph, An augmentation processing step in which the first graph and its global representation, and the second graph and its global representation are input to an augmenter composed of a neural network, respectively, to obtain data augmentation of the first graph and data augmentation of the second graph, A local representation acquisition step involves inputting the data augmentation of the first graph and the data augmentation of the second graph into a second encoder, each composed of a neural network, to obtain the local and global representations of the data augmentation of the first graph and the local and global representations of the data augmentation of the second graph. A similarity calculation step that calculates the similarity between the local representation of the data extension of the first graph and the global representation of the data extension of the second graph, and the similarity between the local representation of the data extension of the second graph and the global representation of the data extension of the first graph, A loss calculation step that calculates a first loss designed such that the value decreases as the similarity between representations based on the same graph increases and as the similarity between representations based on different graphs decreases. A learning process step which involves calculating the gradient of the first loss and updating the parameters of the neural network in the first encoder, the augmenter, and the second encoder, A data augmentation generation method in which a computer performs an output step of outputting local and global representations of the data augmentation of the first graph and local and global representations of the data augmentation of the second graph, when a new graph is made the first graph using the first encoder, the augmenter, and the second encoder that have been trained.

4. A data augmentation generation program for causing a computer to function as a data augmentation generation device according to claim 1 or claim 2.

Images