Data distribution and module contribution degree-based graph classification adversarial training method and system

Abstract

The invention discloses a graph classification adversarial training method and system based on data distribution and module contribution degree. Firstly, a pre-trained graph classification neural network and a training data set are acquired, and an adversarial sample set is generated. Training a graph classification neural network by using the training data set and the generated adversarial sample set based on the constructed distribution-oriented adversarial training loss so as to update network parameters; and calculating the natural robust contribution degree NRC of each layer of the graph classification neural network, selecting the layer with the lowest NRC as a robust redundancy layer, locking parameters of other layers, and only carrying out adversarial training on the robust redundancy layer so as to further improve the tolerance of the model to adversarial disturbance in the image classification process. According to the method, the distribution consistency and the module importance are considered at the same time, and the anti-robustness of the model in the field of image classification is improved.

Description

Technical Field

[0001] This invention belongs to the field of deep learning model security technology, and in particular relates to a graph classification adversarial training method and system based on data distribution and module contribution, which is used to significantly improve the robustness of deep neural networks to adversarial examples without affecting the original accuracy. Background Technology

[0002] Deep neural networks (DNNs) excel in tasks such as image classification and speech recognition, but they are extremely sensitive to adversarial examples: adding minute perturbations to clean samples, imperceptible to humans, can lead to incorrect model outputs. Adversarial training has proven to be one of the most effective defenses; however, existing methods generally suffer from a "robustness-generalization" trade-off, meaning that improving robustness often results in a decrease in original accuracy. Traditional adversarial training frameworks typically update all network parameters uniformly, ignoring the differences in the contribution of different layers (modules) to the loss on natural samples, leading to redundant computation and limiting further improvements in robustness. Furthermore, existing adversarial training strategies often focus on the overall shape of the distribution of clean samples and their corresponding adversarial samples, easily overlooking the numerical differences between categories. Therefore, a novel adversarial training scheme that can simultaneously consider distribution consistency and module importance is urgently needed to improve the adversarial robustness of models in image classification. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a graph classification adversarial training method and system based on data distribution and module contribution.

[0004] In a first aspect, embodiments of this application provide a graph classification adversarial training method based on data distribution and module contribution, comprising the following steps: Step S1: Obtain the pre-trained graph classification neural network and training dataset, wherein the training dataset contains clean samples and their labels.

[0005] Step S2: Apply adversarial perturbation to the clean sample to generate an adversarial sample set.

[0006] Step S3: Construct Distribution-Oriented Adversarial Training Loss (DOAT). The loss includes at least cross-entropy loss, KL divergence loss, and mean squared error loss between intermediate layer features, in order to constrain the consistency between clean samples and adversarial samples in the output distribution and feature space.

[0007] Step S4: Based on the distribution-guided adversarial training loss, the graph classification neural network is trained using the training dataset and the generated adversarial sample set to update the network parameters, so that the model can obtain preliminary robustness without reducing the original accuracy.

[0008] Step S5: Calculate the Natural Robust Contribution (NRC) of each layer (module) of the graph classification neural network. The NRC is determined by applying a perturbation only to the weights of that layer and measuring the maximum change in the loss of clean samples.

[0009] Step S6: Select the layer with the lowest NRC as the robust redundancy layer and lock the parameters of the remaining layers.

[0010] Step S7: Adversarial training is performed only on the robust redundancy layer to further improve the model's tolerance to adversarial perturbations during image classification.

[0011] In one possible implementation, the KL divergence loss is applied to the probability distribution of the model output, and the mean squared error loss is applied to the feature vectors of the intermediate layers of the network.

[0012] In one possible implementation, the NRC calculation uses a random subset of samples. One thousand images are randomly selected from the original training dataset, and corresponding adversarial examples are generated. The perturbation sensitivity of each layer of the graph classification neural network to the output of these images is calculated to accelerate the contribution evaluation.

[0013] In one possible implementation, the adversarial training employs one or a combination of Projected Gradient Descent Adversarial Training (PGD), Trade-off Adversarial Defense (TRADES), or Misclassification Perception Adversarial Training (MART).

[0014] In one possible implementation, the retraining of the robust redundancy layer employs standard cross-entropy loss without introducing additional regularization terms.

[0015] Secondly, embodiments of this application provide a graph classification adversarial training system based on data distribution and module contribution, comprising the following modules: Adversarial example generation module: used to apply perturbations to clean samples to obtain adversarial examples.

[0016] DOAT Loss Calculation Module: Based on Distribution-Directed Adversarial Training Loss (DOAT), it uses the training dataset and the generated adversarial example set to train the graph classification neural network to update the network parameters, enabling the model to achieve initial robustness without reducing the original accuracy.

[0017] The distribution-guided adversarial training loss includes at least cross-entropy loss, KL divergence loss, and mean squared error loss between intermediate layer features, in order to constrain the consistency between clean samples and adversarial samples in the output distribution and feature space.

[0018] NRC Calculation Module: Used to calculate the natural robust contribution (NRC) of each layer (module) of the graph classification neural network. The NRC is determined by applying a perturbation only to the weights of that layer and measuring the maximum change in loss for clean samples.

[0019] The parameter locking and retraining module selects the graph classification neural network layer with the lowest NRC as a robust redundancy layer, locking the parameters of the remaining layers. It then performs adversarial training on the selected robust redundancy layer to further improve the model's tolerance to adversarial perturbations during image classification.

[0020] Control unit: Used to coordinate the adversarial example generation module, DOAT loss calculation module, NRC calculation module and parameter locking and retraining module to perform image classification adversarial training.

[0021] In one possible implementation, the DOAT loss calculation module balances accuracy and robustness by adjusting the adjustable hyperparameters in the distribution-guided adversarial training loss.

[0022] In one possible implementation, the NRC evaluation module employs a subset sampling strategy to reduce computational overhead. It uses random subset samples, randomly selecting one thousand images from the original training dataset and generating corresponding adversarial examples. The module then calculates the perturbation sensitivity of each layer of the graph classification neural network to the output of these images, thereby accelerating the contribution evaluation.

[0023] In one possible implementation, the adversarial training employs one or a combination of Projected Gradient Descent Adversarial Training (PGD), Trade-off Adversarial Defense (TRADES), or Misclassification Perception Adversarial Training (MART).

[0024] In one possible implementation, the parameter locking and retraining module supports backpropagation updates for parameters of only a single layer.

[0025] The beneficial effects of this invention are as follows: 1. By using distribution-level consistency constraints, the robustness of the model to various adversarial attacks is significantly improved; 2. Using module contribution evaluation, only the robust redundancy layer is retrained to maintain the original accuracy to the greatest extent possible; 3. The process is universal. The graph classification adversarial training method of this invention can be seamlessly integrated with existing adversarial training algorithms without modifying the network architecture; 4. The NRC is calculated using subset sampling, which reduces computational overhead and is suitable for large-scale datasets and deep networks. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 : Schematic diagram of the overall process of the method in the embodiment of the present invention.

[0028] Figure 2 The existing framework shows the difference in distribution of adversarial classifiers on clean samples and adversarial samples.

[0029] Figure 3 : Flowchart of NRC evaluation and robust redundancy layer retraining in this embodiment of the invention.

[0030] Figure 4 Comparison table of experimental results of embodiments of the present invention. Detailed Implementation

[0031] The present invention will be further described in detail below with reference to embodiments and accompanying drawings. Those skilled in the art will understand that specific parameters can be adjusted within the scope of the claims without departing from the spirit of the invention.

[0032] This application provides a graph classification adversarial training method based on data distribution and module contribution. Through a two-stage strategy, it simultaneously considers distribution consistency and module importance, improving the model's adversarial robustness in image classification: First stage – Distribution-Oriented Adversarial Training (DOAT): Step 1: Based on the pre-trained graph classification neural network model, generate corresponding adversarial samples for each batch of clean samples (CIFAR10 dataset, 128 samples are randomly selected each time).

[0033] Step 2: Construct the DOAT loss, which includes three components: a) Standard cross-entropy loss This is used to maintain baseline classification accuracy.

[0034] b) KL divergence loss The output probability distribution of clean samples is consistent with that of adversarial samples.

[0035] c) MSE loss of intermediate layer features The model is guided to extract perturbation-invariant features.

[0036] Therefore, the overall DOAT loss term is .

[0037] Step 3: Jointly optimize (α=8.0, β=1.0 in this example) the above loss, so that the model can obtain preliminary robustness while maintaining the original accuracy.

[0038] The second stage—module retraining based on Natural Robust Contribution (NRC), such as... Figure 2 As shown: Step 1: Define the NRC index: Apply a perturbation to the weights of the i-th layer of the graph classification neural network, while keeping the weights of the other layers fixed, and calculate the maximum increase in loss for clean samples as the contribution of that layer to the natural robustness.

[0039] Step 2: Use a random subset sampling strategy to evaluate the NRC scores of each layer (module) of the graph classification neural network to accelerate the evaluation speed, and select the layer with the lowest NRC score as the "robust redundancy layer".

[0040] Step 3: Lock the parameters of the remaining layers and perform adversarial training only on the robust redundancy layer. Use standard cross-entropy loss without additional regularization terms to minimize the impact on generalization and further improve the tolerance of the graph classification neural network to adversarial perturbations.

[0041] This application also provides a graph classification adversarial training system based on data distribution and module contribution, including the following modules: Adversarial example generation module: used to apply perturbations to clean samples to obtain adversarial examples.

[0042] DOAT Loss Calculation Module: Based on Distribution-Directed Adversarial Training Loss (DOAT), it uses the training dataset and the generated adversarial example set to train the graph classification neural network to update the network parameters, enabling the model to achieve initial robustness without reducing the original accuracy.

[0043] The distribution-guided adversarial training loss includes at least cross-entropy loss, KL divergence loss, and mean squared error loss between intermediate layer features, in order to constrain the consistency between clean samples and adversarial samples in the output distribution and feature space.

[0044] NRC Calculation Module: Used to calculate the natural robust contribution (NRC) of each layer in a graph classification neural network. The NRC is determined by applying a perturbation only to the weights of that layer and measuring the maximum change in loss for clean samples.

[0045] The parameter locking and retraining module selects the graph classification neural network layer with the lowest NRC as a robust redundancy layer, locking the parameters of the remaining layers. It then performs adversarial training on the selected robust redundancy layer to further improve the model's tolerance to adversarial perturbations during image classification.

[0046] Control unit: Used to coordinate the adversarial example generation module, DOAT loss calculation module, NRC calculation module and parameter locking and retraining module to perform image classification adversarial training.

[0047] In one possible implementation, the DOAT loss calculation module balances accuracy and robustness by adjusting the adjustable hyperparameters in the distribution-guided adversarial training loss.

[0048] In one possible implementation, the NRC evaluation module employs a subset sampling strategy to reduce computational overhead. It uses random subset samples, randomly selecting one thousand images from the original training dataset and generating corresponding adversarial examples. The module then calculates the perturbation sensitivity of each layer of the graph classification neural network to the output of these images, thereby accelerating the contribution evaluation.

[0049] In one possible implementation, the adversarial training employs one or a combination of Projected Gradient Descent Adversarial Training (PGD), Trade-off Adversarial Defense (TRADES), or Misclassification Perception Adversarial Training (MART).

[0050] In one possible implementation, the parameter locking and retraining module supports backpropagation updates for parameters of only a single layer.

[0051] The graph classification adversarial training method of this invention is plug-and-play with any existing adversarial training algorithm (PGD-AT, TRADES, MART). Experiments show that on CIFAR-10, CIFAR-100, SVHN datasets and networks such as ResNet-18 / 34, PreResNet-18, and WideResNet-28-10, this invention improves the accuracy of PGD-40 attacks by up to 2.99%, improves the robustness of AutoAttack accuracy by up to 1.65%, and keeps the original accuracy loss within 0.6%, effectively mitigating the trade-off between robustness and generalization.

[0052] Example 1: On the CIFAR-10 dataset, a ResNet-18 network was used. Initially, a baseline model was obtained by training with TRADES for 100 epochs. Then, DOAT loss was added, with hyperparameters α=1.0 and β=0.5, and training continued for another 20 epochs. Next, NRC for each layer was calculated using 1000 random samples. The 6th residual block was selected as the robust redundancy module, and the remaining parameters were locked. PGD-10 adversarial training was performed only on this module for 8 epochs. The final model achieved an accuracy of 48.14% under AutoAttack attacks, a 0.64% improvement compared to the original TRADES model, with only a 0.50% decrease in accuracy on clean samples.

[0053] Example 2: On the SVHN dataset, a WideResNet-28-10 network was used, initially trained with MART; the hyperparameters α=0.8 and β=0.4 were set during the DOAT stage; after NRC evaluation, the 4th residual group was selected for retraining. The results showed that the accuracy of PGD-10 attacks improved by 0.86%, while the accuracy of clean samples decreased by 0.74%, verifying the robustness improvement effect of this invention on digit recognition in complex backgrounds.

[0054] The adversarial training method and system proposed in this invention, which is oriented towards data distribution and module contribution, has been proven both theoretically and experimentally to effectively alleviate the robustness-generalization trade-off, providing an efficient and scalable solution for the deployment of deep learning models in real-world security scenarios.

Claims

1. A graph classification adversarial training method based on data distribution and module contribution, characterized in that, Includes the following steps: Step S1: Obtain a pre-trained graph classification neural network and a training dataset, wherein the training dataset contains clean samples and their labels. Step S2: Apply adversarial perturbations to the clean samples to generate an adversarial sample set; Step S3: Construct a distribution-oriented adversarial training loss, which includes at least cross-entropy loss, KL divergence loss, and mean squared error loss between intermediate layer features, in order to constrain the consistency between clean samples and adversarial samples in the output distribution and feature space. Step S4: Based on the distribution-guided adversarial training loss, the graph classification neural network is trained using the training dataset and the generated adversarial sample set to update the network parameters, so that the model can obtain preliminary robustness without reducing the original accuracy. Step S5: Calculate the natural robust contribution (NRC) of each layer of the graph classification neural network. The NRC is determined by applying a perturbation only to the weights of that layer and measuring the maximum change in the loss of clean samples. Step S6: Select the layer with the lowest NRC as the robust redundancy layer and lock the parameters of the remaining layers. Step S7: Adversarial training is performed only on the robust redundancy layer to further improve the model's tolerance to adversarial perturbations during image classification.

2. The graph classification adversarial training method based on data distribution and module contribution as described in claim 1, characterized in that, The KL divergence loss is applied to the probability distribution of the model output, and the mean squared error loss is applied to the feature vectors of the intermediate layers of the network.

3. The graph classification adversarial training method based on data distribution and module contribution as described in claim 1, characterized in that, The NRC calculation uses a random subset of samples. One thousand images are randomly selected from the original training dataset, and corresponding adversarial examples are generated. The sensitivity of each layer of the graph classification neural network to the perturbation of the output of these images is calculated to accelerate the contribution evaluation.

4. The graph classification adversarial training method based on data distribution and module contribution as described in claim 1, characterized in that, The adversarial training employs one or a combination of Projected Gradient Descent (PGD), Trade-off Adversarial Defense (TRADES), or Misclassification Perception (MART).

5. The graph classification adversarial training method based on data distribution and module contribution as described in claim 1, characterized in that, The retraining of the robust redundancy layer uses standard cross-entropy loss and does not introduce additional regularization terms.

6. A graph classification adversarial training system based on data distribution and module contribution, characterized in that, Includes the following modules: Adversarial example generation module: used to apply perturbations to clean samples to obtain adversarial examples; DOAT loss calculation module: Based on distribution-guided adversarial training loss, it uses the training dataset and the generated adversarial example set to train the graph classification neural network to update the network parameters, so that the model can obtain preliminary robustness without reducing the original accuracy; The distribution-guided adversarial training loss includes at least cross-entropy loss, KL divergence loss, and mean squared error loss between intermediate layer features, in order to constrain the consistency between clean samples and adversarial samples in the output distribution and feature space. NRC Calculation Module: Used to calculate the natural robust contribution (NRC) of each layer of the graph classification neural network. The NRC is determined by applying a perturbation only to the weights of that layer and measuring the maximum change in loss for clean samples. Parameter locking and retraining module: This module is used to select the graph classification neural network layer with the lowest NRC as a robust redundancy layer, lock the parameters of the remaining layers, and perform adversarial training on the selected robust redundancy layer to further improve the model's tolerance to adversarial perturbations during image classification. Control unit: Used to coordinate the adversarial example generation module, DOAT loss calculation module, NRC calculation module and parameter locking and retraining module to perform image classification adversarial training.

7. A graph classification adversarial training system based on data distribution and module contribution as described in claim 6, characterized in that, The DOAT loss calculation module balances accuracy and robustness by adjusting the adjustable hyperparameters in the distribution-guided adversarial training loss.

8. A graph classification adversarial training system based on data distribution and module contribution as described in claim 6, characterized in that, The NRC evaluation module employs a subset sampling strategy to reduce computational overhead. It uses random subset samples, randomly selecting one thousand images from the original training dataset and generating corresponding adversarial examples. The module then calculates the perturbation sensitivity of each layer of the graph classification neural network to these image distribution outputs, thereby accelerating contribution evaluation.

9. A graph classification adversarial training system based on data distribution and module contribution as described in claim 6, characterized in that, The adversarial training employs one or a combination of Projected Gradient Descent (PGD), Trade-off Adversarial Defense (TRADES), or Misclassification Perception (MART).

10. A graph classification adversarial training system based on data distribution and module contribution as described in claim 6, characterized in that, The parameter locking and retraining module supports backpropagation updates for parameters of only a single layer.

Images