An adaptive noise-based adversarial sample detection method and system

Abstract

The present application relates to a kind of based on adaptive noise's adversarial sample detection method and system, belong to computer network security technical field.First, the reconstructed residual is obtained as noise distribution by image compression and reconstruction network, then dynamic noise intensity is generated using convolutional neural network, and adaptive noise is formed by combination;By comparing the label consistency and confidence similarity of original sample and noise sample in the output of target model, score is calculated and compared with threshold to realize adversarial sample detection.The present application does not need internal information of target model, can be deployed by relying on a small amount of clean samples training, suitable for various image classification models and adversarial attack types, while ensuring the classification accuracy of clean sample, significantly improve the detection rate of adversarial sample, reduce false positive rate, can be used as third-party detection service adaptation commercial image classification API.

Description

Technical Field

[0001] This invention relates to an adversarial sample detection method and system based on adaptive noise addition, belonging to the field of computer network security technology. Background Technology

[0002] The superiority of deep neural network models enables them to achieve breakthroughs in many fields such as computer vision, natural language processing, and speech recognition, especially in tasks such as face recognition, image classification, and object detection, where they demonstrate accuracy that is close to or even surpasses that of humans.

[0003] However, attackers can generate subtle perturbations that are imperceptible to the human eye through interaction with the model. These perturbations, while maintaining visual naturalness, can effectively mislead the model into producing completely incorrect predictions. This "visually invisible but model-visible" characteristic makes adversarial attacks extremely stealthy and destructive. For example, in facial recognition systems, attackers can bypass authentication using special patterns; in the field of autonomous driving, minor alterations to road signs can lead to fatal accidents; and in medical diagnosis scenarios, subtle perturbations in image data can lead to catastrophic misdiagnosis. Even more serious is the cross-model transferability of adversarial noise, allowing attackers to carry out effective attacks without understanding the details of the target model, thus escalating local security threats into systemic risks and posing a substantial challenge to the reliable deployment of artificial intelligence technologies.

[0004] Existing adversarial example detection techniques suffer from three significant limitations. First, many detection algorithms rely on the internal architectural details and parameter information of the model to achieve effective detection. However, model structure and parameters, as core intellectual property of commercial entities, are often strictly protected and cannot be disclosed, making such methods difficult to deploy in practical applications. Second, existing technologies typically perform binary classification detection against specific types of adversarial attacks (such as FGSM and PGD). Their feature extraction patterns and network structures are highly specific, lacking the ability to generalize to unknown attack types and across attack paradigms, and thus failing to adapt to increasingly diverse adversarial attack methods. Third, in real-world scenarios where detection algorithm access is limited (e.g., only access to model input / output interfaces is available), existing technologies generally suffer from excessively high false positive rates and low detection rates. This severely impacts the practicality and reliability of the detection system, making it difficult for it to function effectively in high-risk real-world environments. Summary of the Invention

[0005] The purpose of this invention is to address the problems and shortcomings of existing technologies, and to solve the technical problems such as strong dependence on black-box scenarios, insufficient cross-attack generalization ability, and low detection accuracy under limited permissions, by creatively proposing an adversarial sample detection method and system based on adaptive noise addition.

[0006] This invention designs a special type of noise that does not affect the classification accuracy of clean samples while mitigating the negative impact of noise-induced model decisions. The noise generation is decomposed into two parts: noise distribution and noise intensity. The noise distribution is obtained through image reconstruction technology, and the corresponding noise intensity is generated through a convolutional neural network. Finally, these components are combined to obtain the desired noise.

[0007] The innovative aspects of this invention include:

[0008] First, a noise distribution generator was designed. It learns the noise distribution pattern of normal samples through an autoencoder structure and generates a constructed noise distribution that is adapted to the image content.

[0009] Secondly, an intensity prediction network was constructed, which uses a lightweight convolutional neural network to dynamically calculate the optimal noise intensity parameters required for each sample.

[0010] Third, the generated noise distribution and noise intensity are combined into the final noise. The difference comparison module analyzes the feature distance between the original sample and the noisy sample in the model output space, and the detection of adversarial samples is achieved based on the threshold judgment.

[0011] This method includes the following steps:

[0012] Step 1: Train the image compression network and the reconstruction network. The compression network maps the image from pixel space to a low-dimensional space, and the reconstruction network maps the low-dimensional features back to pixel space. The residuals before and after image reconstruction are extracted as noise distribution.

[0013] The image compression network architecture includes convolutional kernels and downsampling layers. The reconstruction network maps low-dimensional features to pixel space through deconvolution and upsampling.

[0014] When training the compressed and reconstructed networks, the mean squared error loss function is used as a constraint. The expression for the loss function is as follows:

[0015]

[0016] in, The loss function represents the loss function of the collaboratively trained compression network and reconstruction network. The mean square error before and after image reconstruction reflects whether the information contained in the image has been lost after reconstruction. To compress the network, To reconstruct the network, This is the original image.

[0017] Step 2: Design a noise intensity generation network. Using the noise distribution as input, train the network through a loss function with dual objective constraints to generate one-dimensional noise intensity parameters.

[0018] The loss function expression for the bi-objective constraint is:

[0019]

[0020] in, The loss function representing the noise intensity generation network; This represents the reciprocal of the generated noise intensity value. The optimization objective is to generate as much noise as possible to destroy the effect of counter-noise. This represents the mean square error before and after adding noise. The optimization goal is to minimize the impact on the classification of normal images. For noise intensity, Noise distribution; By balancing two mutually exclusive optimization objectives, the noise intensity is made to fall within a suitable range that neither affects clean samples nor undermines adversarial noise.

[0021] Step 3: Combine noise distribution and noise intensity to obtain adaptive noise, obtain the output results of the original sample and the noisy sample in the target model, extract the output difference features and calculate the score, and realize adversarial sample detection by comparing the score with the preset threshold.

[0022] Among them, the output difference features include label consistency features. Similarity features based on confidence ; This is determined by comparing whether the positions of the maximum values ​​in the output distributions of the original sample and the noisy sample are consistent. This is obtained by calculating the similarity between the output vectors of the two.

[0023] Beneficial effects

[0024] The method of the present invention has the following advantages compared with the prior art:

[0025] 1. This invention is applicable to detecting various adversarial examples generated against image classification models.

[0026] 2. This invention relies less on additional information when detecting adversarial examples. After training with clean samples collected by the model itself, detection can be achieved solely by the model's output signature and confidence level. This pluggable third-party detection method is suitable for providing third-party detection services to existing commercial APIs that provide image classification services.

[0027] 3. This invention designs a noise distribution generation mechanism based on image reconstruction residuals, which can effectively avoid the generated adaptive noise from affecting the classification accuracy of clean images, and make the adaptive noise destroy the negative impact of the noise as much as possible.

[0028] 4. This invention designs a dynamic noise intensity generation mechanism. By further mining the deeper information contained in the noise distribution generated by clean samples and adversarial samples, a specially designed noise intensity is generated through a convolutional neural network. The loss function of the noise intensity generation network is constrained by two optimization objectives: the first objective is to maximize the noise intensity to disrupt adversarial noise; the second objective is to prevent excessive noise intensity from affecting the classification of clean samples.

[0029] 5. This invention extracts differential features from two dimensions: model output confidence score and label, further improving the detection rate of adversarial examples. Attached Figure Description

[0030] Figure 1 This is a flowchart of adversarial example detection based on adaptive noise addition. Detailed Implementation

[0031] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be noted that the implementation of the present invention is not limited to the following embodiments, and any modifications or alterations made to the present invention will fall within the scope of protection of the present invention.

[0032] like Figure 1 As shown, an adversarial example detection method based on adaptive noise addition includes the following steps:

[0033] Step 1: Train an image compression network to map the image from pixel space to a low-dimensional space. Train a reconstruction network to remap the low-dimensional features of the image back to pixel space. The compression and reconstruction networks are combined to complete the image reconstruction process. The residual before and after image reconstruction is used as a noise distribution for subsequent adaptive noise generation.

[0034] Specifically, step 1 may include the following steps:

[0035] Step 1.1: Select a batch of unprocessed image data As a training dataset.

[0036] Step 1.2: Design an image compression network Used to extract images The core content This process filters out unnecessary high-frequency details. Furthermore, the mapping of an image from pixel space to a lower-dimensional space involves dimensionality changes, requiring convolutional operations to achieve this. The compression network architecture consists of convolutional kernels and downsampling layers.

[0037]

[0038] in, .

[0039] Step 1.3: Design and Reconstruct the Network This maps low-dimensional features that retain the main core information of the image back to the pixel space:

[0040]

[0041] in, The reconstructed image is represented by the extracted core features. This is obtained by mapping back to pixel space. The process involves deconvolution and upsampling to remap the low-dimensional features back to pixel space.

[0042] Step 1.4: Train the network.

[0043] To ensure that the compression and reconstruction networks retain the main content of the image while filtering out unnecessary noise and details, the reconstruction effect should be optimized. Using mean squared error loss as a constraint, the image data, after compression and reconstruction, should still maintain a high degree of similarity to the original image. This approach ensures that the low-dimensional features of the image contain as many important features as possible while eliminating as many unnecessary details as possible.

[0044]

[0045] in, This represents the loss function for co-training the compressed network and the reconstructed network. The mean square error before and after image reconstruction reflects whether the information contained in the image has been lost after reconstruction. The smaller the value, the more successfully the network has extracted low-dimensional features that capture key details of the image. And filter out useless pixels and noise.

[0046] For the optimizer chosen during training, Adam can be selected. The Adam optimizer (Adaptive Moment Estimation) is an optimization algorithm based on an adaptive learning rate, which adjusts the learning rate of each parameter by calculating the first and second moment estimates of the gradient.

[0047] Step 1.5: Extract the residuals before and after reconstruction as the noise distribution. :

[0048]

[0049] Step 2: Obtain the noise intensity to form the complete adaptive noise.

[0050] For normal images, the lower the desired noise intensity, the better; for adversarial examples, the higher the desired noise intensity, the better. Noise intensity is a one-dimensional value, while noise distribution is a high-dimensional feature in the pixel domain. Therefore, a convolutional neural network is needed to establish a mapping relationship.

[0051] Specifically, step 2 may include the following steps:

[0052] Step 2.1: Design the noise intensity generation network The input to the noise intensity generation network is the noise distribution. The output is a one-dimensional noise intensity value. By using convolutional neural networks as the main architecture of the network, the need for fast inference based on feature extraction can be met.

[0053] Step 2.2: Design the loss function. It must satisfy the following two dynamically mutually exclusive constraints: To ensure the noise intensity is large enough to destroy the adversarial noise, the first part of the loss function consists of the reciprocal of the noise intensity; to prevent the noise intensity from being too large and thus compromising the classification accuracy of clean samples in the network, the second part of the loss function consists of the mean squared error before and after adding noise.

[0054]

[0055] in, The loss function representing the noise intensity generation network; This represents the reciprocal of the generated noise intensity value. The optimization objective is to generate as much noise as possible to destroy the effect of counter-noise. This represents the mean square error before and after adding noise. The optimization goal is to minimize the impact on the classification of normal images. By balancing two mutually exclusive optimization objectives, the noise intensity is made to fall within a suitable range that neither affects clean samples nor undermines adversarial noise.

[0056] Step 2.3: Train the network.

[0057] The Adam optimizer can be selected during the training process.

[0058] Step 3: Combine the noise distribution and noise intensity obtained in Step 1 and Step 2 to obtain adaptive noise. By comparing the target model before and after adding noise. The output changes to calculate a score, and adversarial example detection is achieved by determining the relationship between the score and a threshold.

[0059] Specifically, step 3 may include the following steps:

[0060] Step 3.1: Combine noise distribution and noise intensity to obtain adaptive noise. :

[0061]

[0062] Step 3.2: Obtain the output of the target model before and after applying adaptive noise. and .

[0063] Step 3.3: Compare whether the positions of the maximum values ​​in the output distribution are the same, as a feature. Value:

[0064]

[0065] Step 3.4: Compare the similarity scores of the two outputs as features. Value:

[0066]

[0067] Step 3.5: Synthesis and The value of the score is used as the final score. :

[0068]

[0069] Step 3.6: Set a threshold based on clean samples Calculated Then, compare it with the threshold. Compare, if it is greater than If it is not adversarial, it is considered an adversarial sample; otherwise, it is considered a clean sample.

[0070] Furthermore, this invention proposes an adversarial example detection system based on adaptive noise generation to implement the above method, including a noise distribution generation module, a noise intensity generation module, and an adversarial example detection module.

[0071] Noise Distribution Generation Module: Responsible for generating a noise distribution for a specific input image. This module extracts key details from the image through image reconstruction while filtering out some unnecessary details. The module uses the residual as the final noise distribution. Since adversarial noise is essentially a high-frequency signal, a portion of it is filtered out during this process. Therefore, the residual before and after image reconstruction achieves the effect of not affecting normal image classification while simultaneously perceiving and disrupting the adversarial noise distribution.

[0072] The noise intensity generation module is responsible for generating appropriate noise intensity based on the noise distribution provided by the noise distribution generation module. Excessive noise intensity will affect the classification accuracy of normal images; insufficient noise intensity will not effectively combat adversarial noise. Therefore, the noise intensity generation module designs mutually exclusive optimization objectives and utilizes a convolutional neural network to further mine the characteristics of the noise distribution to generate the final noise intensity.

[0073] Adversarial Example Detection Module: This module is responsible for detecting adversarial examples based on the generated noise distribution and intensity. It first combines the noise distribution and intensity generated by the first two modules to form the final noise. Then, it determines whether the input is an adversarial example by observing the changes in the output label and output confidence of the input sample before and after adding noise in the target model.

[0074] The output of the noise distribution generation module is connected to the input of the noise intensity generation module. The output of the noise intensity generation module is connected to the input of the adversarial example detection module.

[0075] Example

[0076] An adversarial example detection method based on adaptive noise addition includes the following steps:

[0077] Step 1: Select four commonly used public datasets in the field of image classification, including different numbers of image categories and different image resolutions.

[0078] The CIFAR-10 dataset contains images of size 3x32x32, representing ten categories, with 50,000 training images and 10,000 test images. The ImageNet-10 dataset contains images of size 3x256x256, representing ten categories, with 50,000 training images and 10,000 test images. The ImageNet-1000 dataset contains images of size 3x256x256, representing 1000 categories, with 200,000 training images and 20,000 test images. The CelebA dataset contains images of size 3x256x256, representing 100 categories, with 40,000 training images and 10,000 test images. Detailed information about the datasets is shown in Table 1.

[0079] Table 1

[0080]

[0081] Step 2: Use the training images to train the image compression network, the image reconstruction network, and the noise intensity generation network.

[0082] Specifically, for the target model to be protected, ResNet50 can be selected. The classification accuracy of the trained ResNet50 network on the test images of the above four datasets is 94.75%, 98.75%, 78.80%, and 95.19%, respectively.

[0083] Step 3: After the network structure is trained, generate adversarial samples against ResNet50 for the test images as test data.

[0084] There are many types of adversarial attack methods. In this embodiment, four classic methods, PGD, CW, Hybrid, and HJSA, as well as two state-of-the-art attacks, AutoAttack and DSA, are selected as attack methods. For each test image, a corresponding adversarial sample is generated.

[0085] Step 4: For comparison schemes, select classic adversarial example detection methods MagNet, FS and RS, as well as adversarial example detection methods CNet, MIAED and Beyond.

[0086] Step 5: Evaluate the effectiveness of the inventive method.

[0087] The method proposed in this invention is compared with the aforementioned six methods (MagNet, FS, RS, CNet, MIAED, and Beyond) in two scenarios: one where the detector has access to all training data of the target model, and the other where only a portion of the data is available. The comparison metrics are the detection rate of adversarial examples and the false positive rate of clean examples. Table 2 shows the TPR (%) for adversarial examples and the FPR (%) for clean examples when the detector has access to the target model's training data.

[0088] Table 2

[0089]

[0090] As shown in Table 2, when the detector has access to the target model's training data for detection, this method achieves the highest detection rate for adversarial examples and minimizes the false positive rate for clean samples, outperforming the comparison scheme. On the four selected datasets, this method achieves the greatest performance improvement on ImageNet-1000 and CelebA. This is because these two datasets have many image categories and high resolution, representing relatively complex image classification tasks. Some methods relying on simple representation extraction (such as FS) experience a significant performance drop on complex datasets. However, this method maintains good performance even on complex datasets. Table 3 shows the TPR (%) for adversarial examples and the FPR (%) for clean samples when the detector cannot obtain the target model's training data.

[0091] Table 3

[0092]

[0093] As shown in Table 3, when the detector cannot use the training data of the target model for detection, this method achieves the highest detection rate for adversarial examples and minimizes the impact of false positives on clean samples, outperforming the comparison scheme. Despite some restrictions on the detector's permissions, the performance degradation is not significant.

Claims

1. A method for detecting adversarial examples based on adaptive noise addition, characterized in that, Includes the following steps: Step 1: Train the image compression network and the reconstruction network. The compression network maps the image from the pixel space to a low-dimensional space, and the reconstruction network maps the low-dimensional features back to the pixel space. The residual before and after image reconstruction is extracted as the noise distribution. Step 2: Design a noise intensity generation network. Using the noise distribution as input, train the network through a loss function with dual objective constraints to generate one-dimensional noise intensity parameters. Step 3: Combine noise distribution and noise intensity to obtain adaptive noise, obtain the output results of the original sample and the noisy sample in the target model, extract the output difference features and calculate the score, and realize adversarial sample detection by comparing the score with the preset threshold.

2. The adversarial example detection method based on adaptive noise addition as described in claim 1, characterized in that, In step 1, the architecture of the image compression network includes convolutional kernels and downsampling layers. The reconstruction network achieves the mapping of low-dimensional features to pixel space through deconvolution and upsampling.

3. The adversarial example detection method based on adaptive noise addition as described in claim 1, characterized in that, In step 1, when training the compressed network and the reconstructed network, the mean squared error loss function is used as a constraint. The expression for the loss function is: in, The loss function represents the loss function of the collaboratively trained compression network and reconstruction network. The mean square error before and after image reconstruction reflects whether the information contained in the image has been lost after reconstruction. To compress the network, To reconstruct the network, This is the original image.

4. The adversarial example detection method based on adaptive noise addition as described in claim 1, characterized in that, In step 2, the loss function expression for the bi-objective constraint is: in, The loss function representing the noise intensity generation network; This represents the reciprocal of the generated noise intensity value. The optimization objective is to generate as much noise as possible to destroy the effect of counter-noise. This represents the mean square error before and after adding noise. The optimization goal is to minimize the impact on the classification of normal images. For noise intensity, Noise distribution; By balancing two mutually exclusive optimization objectives, the noise intensity is made to fall within a suitable range that neither affects clean samples nor undermines adversarial noise.

5. The adversarial example detection method based on adaptive noise addition as described in claim 1, characterized in that, In step 3, the output difference features include label consistency features. Similarity features based on confidence ; This is determined by comparing whether the positions of the maximum values ​​in the output distributions of the original sample and the noisy sample are consistent. This is obtained by calculating the similarity between the output vectors of the two. Step 3.1: Distribute the noise and noise intensity Combining them, we obtain adaptive noise. : Step 3.2: Obtain the output of the target model before and after applying adaptive noise. and , To compress the network, Original image; Step 3.3: Compare whether the positions of the maximum values ​​in the output distribution are the same, as a feature. Value: Step 3.4: Compare the similarity scores of the two outputs as features. Value: Step 3.5: Synthesis and The value of the score is used as the final score. : Step 3.6: Set a threshold based on clean samples Calculated Then, compare it with the threshold. Compare, if it is greater than If it is not adversarial, it is considered an adversarial sample; otherwise, it is considered a clean sample.

6. The adversarial example detection method based on adaptive noise addition as described in claim 1, characterized in that, The Adam optimizer is used to adjust network parameters when training the compression network, reconstruction network, and noise intensity generation network.

7. A method for detecting adversarial examples based on adaptive noise addition, implementing the method of claim 1, characterized in that, It includes a noise distribution generation module, a noise intensity generation module, and an adversarial example detection module; Noise distribution generation module: responsible for generating noise distribution for a specific input image; this module extracts key details from the image while filtering out some unnecessary details through image reconstruction; this module uses the residual as the final noise distribution; Noise intensity generation module: Responsible for generating appropriate noise intensity based on the noise distribution provided by the noise distribution generation module; The noise intensity generation module further mines the characteristics of the noise distribution by designing mutually exclusive optimization objectives and using convolutional neural networks to generate the final noise intensity; Adversarial Example Detection Module: This module is responsible for detecting adversarial examples based on the generated noise distribution and intensity. It first combines the noise distribution and intensity generated by the first two modules to form the final noise. It then determines whether the input is an adversarial example by the degree of change in the output label and output confidence of the input sample before and after adding noise. The output of the noise distribution generation module is connected to the input of the noise intensity generation module; the output of the noise intensity generation module is connected to the input of the adversarial example detection module.

Images