Graph network backdoor attack defense method based on contrastive learning

Abstract

The application provides a graph network backdoor attack defense method based on contrast learning. The method constructs a graph encoder, replaces an encoding part of a trained graph neural network model, constructs a defense model having a defense function against backdoor attacks, and realizes graph network backdoor attack defense. The method can directly destroy a trigger structure inserted in graph data, causes the trigger structure to fail to achieve the desired effect, but does not affect normal samples. The model can still exhibit the desired performance for input normal samples.

Description

Technical Field

[0001] This invention relates to a graph network backdoor attack defense method based on contrastive learning. Background Technology

[0002] Most systems in real life can be represented using graph data, such as social networks, communication networks, biological networks, and smart grids. As deep learning has gradually become a research hotspot and mainstream development direction in the field of artificial intelligence, graph neural networks have developed rapidly, and more and more deep neural networks are being used to solve graph evolution tasks, such as graph classification, node classification, and link prediction.

[0003] While graph neural networks (Graph Neural Networks) perform downstream tasks with high quality, their robustness is also a major concern. A significant portion of the success of Graph Neural Network models stems from the support of large amounts of data. However, several backdoor attack methods targeting the model training phase have been proposed. Backdoor attacks occur during the training phase, where attackers train the model using training data with pre-defined triggers. This allows the model to respond to data inputs with trigger embeddings in a highly predictable manner during the application phase, resulting in pre-set results. For other normal samples input into the model, it functions correctly. Once triggers are set during training, the model essentially provides a backdoor for attackers. Inputting data with trigger embeddings during the model's application phase can lead to extremely serious consequences. Summary of the Invention

[0004] To overcome the aforementioned shortcomings of existing technologies, this invention provides a graph network backdoor attack defense method based on contrastive learning. Specifically, during the model testing phase, the input model data is preprocessed, and contrastive learning is used to maximize the features of the graph network itself and reduce the features of data triggers, thereby achieving a defensive effect.

[0005] The technical solution adopted by this invention to solve its technical problem is:

[0006] A graph network backdoor attack defense method based on contrastive learning is proposed. This method constructs a graph encoder to replace the encoding part of a trained graph neural network model, thereby constructing a defense model capable of defending against backdoor attacks and achieving graph network backdoor attack defense. Specifically, it includes:

[0007] An augmented graph is obtained using the encoding part of a trained graph neural network model; the augmented graph includes a first augmented graph and a second augmented graph, wherein the adjacency matrix of the first augmented graph is composed of the adjacency matrix of the original graph and an adjacency matrix mask; the feature matrix of the second augmented graph is composed of the feature matrix of the original graph and a feature matrix mask.

[0008] The training dataset for the graph encoder is constructed based on the acquired augmented graphs. The samples in the training dataset are used as the input to the graph encoder, and the embedding features of each augmented graph are output. The similarity between each pair of embedding features is calculated. The graph encoder is trained with the goal of maximizing the similarity of the embedding features of positive samples and minimizing the similarity of the embedding features of negative samples. Finally, the trained graph encoder is obtained. Positive samples are a pair of first and second augmented graphs constructed from the same original graph, and negative samples are a pair of first and second augmented graphs constructed from different original graphs.

[0009] The trained graph encoder is combined with the decoding module of the trained graph neural network model to obtain a defense model that can defend against backdoor attacks.

[0010] Furthermore, the process of obtaining the enhanced graph using the encoding portion of the trained graph neural network model specifically involves:

[0011] The first and second enhanced graphs are initialized, wherein the adjacency matrix mask of the first enhanced graph and the feature matrix mask of the second enhanced graph are both all-one matrices.

[0012] The initial first augmented graph and the original graph constituting the first augmented graph are used as samples to be input into the encoding part of the trained graph neural network model. The encoding part of the trained graph neural network model outputs the embedding vector of each sample. With the goal of minimizing the loss of the embedding vectors of the first augmented graph and the original graph constituting the first augmented graph, the adjacency matrix mask of each first augmented graph is updated, and finally the first augmented graph is obtained.

[0013] The initialized second augmented map and the original map constituting the second augmented map are used as samples to be input into the encoding part of the trained graph neural network model. The encoding part of the trained graph neural network model outputs the embedding vector of each sample. With the goal of minimizing the loss of the embedding vectors of the second augmented map and the original map constituting the second augmented map, the feature matrix mask of each second augmented map is updated, and finally the second augmented map is obtained.

[0014] Furthermore, the loss of the first enhanced graph and the embedding vector of the original graph constituting the first enhanced graph, and the loss of the second enhanced graph and the embedding vector of the original graph constituting the second enhanced graph, are adopted using normalized temperature cross-entropy loss.

[0015] Furthermore, the graph encoder is composed of a two-layer GCN.

[0016] Furthermore, the graph encoder is trained with the objective of maximizing the similarity of the embedding features of positive samples and minimizing the similarity of the embedding features of negative samples. The objective function is:

[0017]

[0018]

[0019] Where, sim(z) i ,z j ) represents the embedding feature z of the positive sample. i ,z j Similarity, sim(z) i ,z k ) represents the embedding feature z of the negative sample. i ,z k The similarity is given by K, where K is the number of original images and Γ represents the temperature parameter.

[0020] Furthermore, the original graphs constituting the first and second enhanced graphs originate from the training dataset of the trained graph neural network model.

[0021] Furthermore, the trained graph neural network model is obtained through training on the acquired graph data; wherein the graph data is obtained by processing the collected node data; the structural information of the graph data is represented by the adjacency matrix A. um This means that if there is an edge between nodes u and m, then at their corresponding adjacency matrix position e... um The value is 1, and there is no edge e. um Its corresponding value is 0, and each node has a feature matrix X~U(0,1) from the same distribution.

[0022] The technical concept of this invention is as follows: Backdoor attacks are triggered by using subgraph structures as triggers, causing the neural network model with the trigger set to output incorrect results. Therefore, this invention proposes a graph network backdoor attack defense method based on contrastive learning. First, data augmentation is performed by changing attributes and links to optimize an augmented network that is as similar as possible to the original network. Second, the augmented network is passed through a GNN-based encoder, and the encoder parameters are optimized through contrastive learning to make the features of positive samples as similar as possible and the features of negative samples as different as possible. Finally, after the GNN-based encoder is trained, during the model testing phase, the input data first passes through the GNN encoder, and then the encoder output is input into the model to complete the downstream task. The main idea is that the trigger is placed in the graph network. For the overall network, the trigger has its own special structural characteristics, which may be inconsistent with the overall network topology. Therefore, contrastive learning is used to learn its own important features, minimizing the impact of the trigger's features on the task.

[0023] The main advantages of this invention are as follows: it proposes a defense method against most existing backdoor attack methods. This method can directly destroy the trigger structure inserted in the graph data, preventing it from achieving its intended effect, but it does not affect normal samples; the model can still exhibit its expected performance for input normal samples. Attached Figure Description

[0024] Figure 1 This is a flowchart of the training process for a graph encoder based on GNN.

[0025] Figure 2 This is a framework diagram of a graph network backdoor attack defense method based on contrastive learning. Detailed Implementation

[0026] This invention provides a defense method against backdoor attacks in graph classification. This method constructs a graph encoder to replace the encoding part of a trained graph neural network model, thereby creating a defense model capable of defending against backdoor attacks. Correspondingly, this invention is also a method for constructing a graph network defense model. Graph data is widely present in the real world, and graph classification has many key applications in biology, chemistry, and informatics, such as citation network classification, molecular attribute determination, and new drug discovery. The invention is further described below with reference to the accompanying drawings and specific embodiments. This invention uses graph classification in the field of biochemistry as an example to describe the backdoor defense method. Correspondingly, this invention is also a method for constructing a molecular compound classification defense model based on graph networks. Please refer to [link to relevant documentation]. Figure 1 , Figure 2 Includes the following steps:

[0027] S1. Construct a graph neural network model for classifying molecular compounds and a training dataset for the graph neural network model.

[0028] S11, Obtain the data for model M oracle Training dataset oracle In this embodiment, the MUTAG dataset is used directly. Each graph network sample represents a nitro compound molecule, where atoms are nodes and the chemical bonds between atoms are links in the graph network. Each sample has a corresponding label, which is mutagenic aromatic and heteroaromatic, represented by 0 and 1. In practical applications, this dataset can be used to train a graph neural network model for predicting nitro compound molecules, efficiently classifying molecular compounds and reducing the manpower and time costs.

[0029] In addition, for different application scenarios, datasets can be collected or gathered for training based on actual needs. Examples include the COLLAB dataset, the Twitter dataset, and the Bitcoin dataset. For different datasets, the nodes and edges have corresponding meanings, and each graph G... i Each has a corresponding label y for its category. i In general, graph data consists of nodes and edges, where graph data is represented by G, and the structural information of graph data is represented by the adjacency matrix A. um This means that if there is an edge between nodes u and m, then at their corresponding adjacency matrix position e... um The value is 1, and there is no edge e. um Its corresponding value is 0, and each node has a feature matrix X~U(0,1) from the same distribution.

[0030] S12, the graph neural network model M oracle In this invention, the downstream task performed by the model is a graph-based molecular compound classification task, in which the acquired training data is input into the graph neural network model M. oracle The process involves training the graph classifier f: G → {0, 1, ..., y}, ultimately resulting in a fully trained graph classifier. i}, where G is the corresponding input sample, {0,1,...,y i} represents the predicted label for the corresponding classifier.

[0031] S2. Using the encoding part of the trained graph neural network model, contrastive learning is employed to obtain an enhanced graph.

[0032] S21. Obtain the original graph for constructing the augmented graph. In this embodiment, the graph is obtained from the training dataset Data in step S12. oracle Randomly select a certain percentage (e.g., 10%, 15%, etc.) of the data to form a dataset. defense There are K graphs in total that serve as the original graphs for the augmentation graph;

[0033] S22, Transfer the dataset Data defense For each graph G, add an adjacency matrix A with all 1s to the adjacency matrix mask. adj The initial adjacency matrix A is obtained. mask For each sample, there is an adjacency matrix A and an adjacency matrix mask Mask. adj and the initial adjacency matrix A mask The dimensions are the same, and the formula is as follows.

[0034] A mask =A+Mask adj

[0035] S23. Input the original image of each sample into model M. oracle The embedding vector h of the original graph is obtained from the initial adjacency matrix A. mask The first augmented graph input model M is obtained from A in the original graph of the substitute sample. oracle The embedding vector h of the first augmented graph is obtained. mask The goal is to minimize the loss of the embedding vectors of the first augmented graph and the original graphs that constitute the first augmented graph. The adjacency matrix mask of each first augmented graph is updated to obtain the final first augmented graph. In this embodiment, the embedding vector h of the original graph and the embedding vector h of the first augmented graph are used as the weights. mask The distance between them is used as the loss. adj The first and second neighbor embedding vectors h are calculated using the L2 normal form. mask Distance loss between adj For example, the formula is as follows:

[0036]

[0037] Where the superscript i represents the number of the original graph; ||*||2 represents the L2 norm.

[0038] Specifically, this embodiment uses the Adam optimizer to optimize the adjacency matrix mask. adj Update to reduce loss adj The size of the dataset is determined by limiting the size of the perturbation to the adjacency matrix mask. defense In the K graphs G, the last updated Mask is used to determine the values. adj Enhanced A mask The resulting K first augmented graphs

[0039] S24. Similarly, the dataset Data defense For each graph G, add a feature matrix X of all 1s to the feature matrix mask. feature The initialized feature matrix X is obtained. mask For each sample, there is a feature matrix X and a feature matrix mask Mask. feature and the initialized eigenma matrix X mask The dimensions are the same, and the formula is as follows.

[0040] X mask =X+Mask feature

[0041] S25. Input the original image of each sample into model M. oracle The embedding vector l of the original image is obtained from the initialization of the feature matrix X. maskThe feature matrix X from the substitute sample is used to initialize the second augmented graph input model M. oracle The embedding vector l of the second augmented graph is obtained. mask The goal is to minimize the loss of the embedding vectors of the original graph that constitutes the second augmented graph. The adjacency matrix mask of each second augmented graph is updated to obtain the final second augmented graph. Similarly, in this embodiment, a method is used to measure the difference between the embedding vector l of the original graph and the embedding vector l of the second augmented graph. mask Distance calculation loss feature The embedding vector l of the original graph and the embedding vector l of the second augmented graph are calculated using the L2 paradigm. mask Taking the distance as an example, the formula is as follows:

[0042]

[0043] For the same original graph, the embedding vector h and the embedding vector l should be the same.

[0044] Specifically, this embodiment uses the Adam optimizer to optimize the feature matrix mask. feature Update to reduce loss feature The size of the dataset is determined by limiting the size of the perturbation to the feature matrix mask. defense In the K graphs G, the last updated Mask is used to determine the values. feature Enhanced X mask K second-enhanced graphs were obtained.

[0045] S3. Construct and optimize a GNN-based graph encoder.

[0046] S31. This invention requires a graph encoder based on a GNN to perform graph encoding on augmented data. Preferably, a multi-layer GCN structure can be used as the graph encoder. This structure can extract and fuse node features and structural information in the graph, embedding each node into a high-dimensional vector space. Taking a two-layer GCN as an example, the specific operation method is as follows:

[0047]

[0048] Where A represents the adjacency matrix of the graph, and X is the characteristic matrix. I represents the normalized adjacency matrix. N It is the identity matrix. This is the degree matrix of the nodes in the graph, which is a diagonal matrix, and its i-th diagonal element is... A∈R N×N , X∈R N×M W (0) ∈R M×H and W (1) ∈RH×F It is the weight matrix of a two-layer graph convolutional layer, Z∈R N×F Let N represent the hidden features of each node output by the graph convolutional layer, where N represents the number of nodes in the graph, H represents the dimension of the hidden layer, F represents the dimension of the output node embedding vector, M represents the dimension of the node features, and σ is the ReLU activation function.

[0049] S32. Construct the training dataset for the graph encoder based on the acquired augmented graph. The dataset Data is obtained from step S2. defense The augmented data consists of 2K graph networks, meaning each original graph in the dataset has two augmented graphs. All these augmented graphs form the training dataset for the graph encoder. Each sample from the training dataset is fed into the GNN-based graph encoder, which outputs the embedding feature z for each augmented graph. The similarity between pairwise embedding features is calculated, and the graph encoder is trained with the goal of maximizing the similarity of embedding features for positive samples and minimizing the similarity of embedding features for negative samples. The result is a well-trained graph encoder.

[0050] Specifically, a pair of first and second augmented images from the same original image are considered positive samples, while a pair of first and second augmented images not from the same original image are considered negative samples. In this embodiment, cosine similarity is used to calculate the similarity of pairwise embedded features. Taking the similarity between the embedding vectors of a pair of positive samples' augmented images as an example, the formula is as follows:

[0051]

[0052] Among them, z i ,z j This represents the embedding features of positive samples. ||*|| represents the magnitude of the vector.

[0053] In this embodiment, using the standardized temperature-scale cross-entropy loss (NT-Xent), the loss for a pair of positive samples i, j can be expressed as:

[0054]

[0055] Where Γ represents the temperature parameter, and the final loss is calculated by considering all positive samples.

[0056]

[0057] By minimizing the loss, we can maximize the similarity of the embedded features of positive samples and minimize the similarity of the embedded features of negative samples.

[0058] When training and optimizing the encoder parameters of a GNN, a mini-bach gradient descent (MBGD) training method is adopted. Each time, a batch of data is randomly selected from the training set for model training. This avoids the training oscillations caused by stochastic gradient descent (SGD) and the excessive resource consumption of batch gradient descent (BGD). The training objective is to adjust the network's structural parameters through forward and backward propagation of gradients, continuously reducing the model's loss function value, and finally obtaining a well-trained graph encoder.

[0059] S4. Combining the trained graph encoder with the decoding module of the trained graph neural network model yields a molecular compound classification defense model that defends against backdoor attacks.

[0060] When using this defense model, for the neural network model M that needs to be fed in... oracle Data D for downstream tasks real First, D real The graph network is input into the GNN-based encoder to obtain its embedded features z.

[0061] Then, the embedded feature z is fed into the decoding module of the trained graph neural network model to obtain the corresponding task result.

[0062] This invention augments data by modifying the properties and links of a graph network, optimizing it to create an augmented network as similar as possible to the original network. Secondly, the augmented network is passed to a GNN-based encoder, and its parameters are optimized through contrastive learning to ensure that the features of positive samples are as similar as possible, while the features of negative samples are as different as possible. The main idea is to embed triggers into the graph network. For the overall network, triggers have their own unique structural features, which may appear inconsistent with the overall network topology. Therefore, contrastive learning is used to learn their important features, minimizing the impact of trigger features on the task and thus acting as a defense against backdoor attacks, improving the model's security.

[0063] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A graph network backdoor attack defense method based on contrastive learning, characterized in that, The method realizes the defense of the graph network backdoor attack by constructing a graph encoder to replace the encoding part of the trained graph neural network model and constructing a defense model with defense against the backdoor attack, and specifically comprises the following steps: The encoding part of the trained graph neural network model is used to obtain an enhanced graph; the enhanced graph comprises a first enhanced graph and a second enhanced graph, wherein the adjacency matrix of the first enhanced graph is composed of the adjacency matrix of an original graph and an adjacency matrix mask, and the feature matrix of the second enhanced graph is composed of the feature matrix of the original graph and a feature matrix mask; A training data set of the graph encoder is constructed based on the obtained enhanced graph, samples of the training data set are taken as the input of the graph encoder, the embedding features of each enhanced graph are output, the similarity of the embedding features is calculated, and the graph encoder is trained with the goal of maximizing the similarity of the embedding features of positive samples and minimizing the similarity of the embedding features of negative samples, and finally a trained graph encoder is obtained; wherein the positive sample is a pair of the first enhanced graph and the second enhanced graph constructed from the same original graph, and the negative sample is a pair of the first enhanced graph and the second enhanced graph constructed from different original graphs; The trained graph encoder and the decoding module of the trained graph neural network model are combined to obtain a defense model with defense against the backdoor attack.

2. The method of claim 1, wherein, The encoding part of the trained graph neural network model is used to obtain an enhanced graph; specifically: The first enhanced graph and the second enhanced graph are initialized, wherein the initialized adjacency matrix mask of the first enhanced graph and the initialized feature matrix mask of the second enhanced graph are all full-matrix; The initialized first enhanced graph and the original graph constituting the first enhanced graph are input into the encoding part of the trained graph neural network model as samples, the encoding part of the trained graph neural network model outputs the embedding vector of each sample, and the adjacency matrix mask of each first enhanced graph is updated with the goal of minimizing the loss of the embedding vector of the first enhanced graph and the original graph constituting the first enhanced graph, and finally the first enhanced graph is obtained; The initialized second enhanced graph and the original graph constituting the second enhanced graph are input into the encoding part of the trained graph neural network model as samples, the encoding part of the trained graph neural network model outputs the embedding vector of each sample, and the feature matrix mask of each second enhanced graph is updated with the goal of minimizing the loss of the embedding vector of the second enhanced graph and the original graph constituting the second enhanced graph, and finally the second enhanced graph is obtained.

3. The method of claim 2, wherein, The loss of the embedding vector of the first enhanced graph and the original graph constituting the first enhanced graph and the loss of the embedding vector of the second enhanced graph and the original graph constituting the second enhanced graph adopt normalized temperature cross-entropy loss.

4. The method of claim 1, wherein, The graph encoder is composed of a double-layer GCN.

5. The method of claim 1, wherein, The graph encoder is trained with the goal of maximizing the similarity of the embedding features of positive samples and minimizing the similarity of the embedding features of negative samples, and the objective function is: where sim(z i ,z j ) represents the similarity of the embedding features z i ,z j of the positive samples, sim(z i ,z k ) represents the similarity of the embedding features z i ,z k of the negative samples, K is the number of original graphs, and Γ represents the temperature parameter.

6. The method of claim 1, wherein, The original graphs constituting the first enhanced graph and the second enhanced graph are derived from the training data set of the trained graph neural network model.

Images