A training method for physical light backdoor attacks facing artificial intelligence security

Abstract

The application belongs to the field of artificial intelligence security, and discloses a training method for physical light backdoor attack for artificial intelligence security, comprising the following steps: performing light backdoor attack on a target object, generating corresponding light triggers on the target object according to light colors, and generating backdoor image data based on the light triggers; obtaining clean image data, and respectively constructing training sets based on the backdoor image data and the clean image data; the clean image data is original image data without light triggers; constructing a backdoor model, the backdoor model is a deep learning model, training the backdoor model based on the training sets to obtain a trained backdoor model; constructing a test set, evaluating the trained backdoor model based on the test set to obtain attack success rate data and clean accuracy rate data of the light backdoor attack. The technical scheme disclosed by the application realizes more covert physical backdoor attack while having a higher attack success rate.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence security, and in particular relates to a training method for physical light backdoor attacks aimed at artificial intelligence security. Background Technology

[0002] Deep neural networks have been widely used in fields such as facial recognition, autonomous driving, and natural language processing. However, deep neural networks are vulnerable and easily attacked by adversaries. Among these attacks, adversarial examples are the most well-known. Adversarial examples add subtle perturbations to images to deceive deep neural networks, causing them to be misclassified.

[0003] Backdoor attacks are another threat to deep neural networks. Unlike adversarial examples, backdoor attacks focus more on the model's training phase. They aim to poison a small subset of training samples and launch an attack while the model is being trained. Attackers share the trained backdoor model online, which can pose security risks when users download and use it.

[0004] Taking image classification as an example, backdoor attacks are launched using poisoned training sets. Attackers add carefully crafted triggers to a portion of the clean training set; these images with added triggers are called backdoor images. Furthermore, the labels of the backdoor training set are set as the target labels, and then the clean and poisoned training sets are mixed to train the model. The model trained in this way performs normally when predicting the clean test set, but will misclassify the backdoor test set. Backdoor attacks are categorized into poisoned label backdoor attacks and clean label backdoor attacks based on whether the target label matches the original content of the backdoor image.

[0005] Digital backdoor attacks: Currently, most research on backdoor attacks focuses on digital backdoor attacks, which involve adding triggers to images using digital modification methods. The backdoor trigger can be a small patch, a pixel, or even a mixture of images.

[0006] Physical backdoor attacks use real-world objects as triggers instead of digitally generated ones, making the images appear more natural. For example, a physical trigger might be sunglasses, a headscarf, or a sticker on a traffic sign.

[0007] Current problems with physical backdoor attacks:

[0008] (1) In some cases, it is difficult for attackers to directly access the target object to deploy triggers, such as traffic signs on high poles;

[0009] (2) It is unnatural to stick stickers directly on the target object, which is likely to arouse suspicion. Summary of the Invention

[0010] The purpose of this invention is to provide a training method for physical light backdoor attacks aimed at artificial intelligence security, so as to solve the problems existing in the prior art.

[0011] To achieve the above objectives, this invention provides a training method for physical light backdoor attacks aimed at artificial intelligence security, comprising:

[0012] A backdoor attack is performed on the target object using light. A corresponding light trigger is generated on the target object based on the light color, and backdoor image data is generated based on the light trigger.

[0013] Obtain clean image data, and construct training sets based on the backdoor image data and the clean image data respectively; the clean image data is the original image without the generation of light triggers;

[0014] Construct a backdoor model, wherein the backdoor model is a deep learning model, and train the backdoor model based on the training set to obtain the trained backdoor model;

[0015] A test set is constructed, and the trained backdoor model is evaluated based on the test set to obtain the attack success rate data and clean accuracy data of the light backdoor attack.

[0016] Optionally, the light trigger includes three colors: blue, green, and red.

[0017] Optionally, the process of generating the backdoor image data includes:

[0018] After a light trigger is generated on the target object, the target object from which the light trigger was generated is photographed by a camera to obtain the backdoor image data.

[0019] Optionally, the clean image data is a CTSRD traffic sign dataset that has not been subjected to a light backdoor attack.

[0020] Optionally, the process of constructing the training set includes:

[0021] The backdoor image data is used as the backdoor training set X. b and the backdoor training set X b All images in the clean image data are labeled as target labels, and the clean image data is used as the clean training set X. c For the clean training set X c and backdoor training set X b By performing a combined analysis, the training set X is obtained. p ;

[0022] Wherein, the training set X is obtained. p The calculation formula is:

[0023] X p =X b ∪X c .

[0024] Optionally, the trained backdoor model f θ for:

[0025] f θ (x) = y, f θ (T p )=y′

[0026] T p (x i ,t p ,m i )=(1-m i )·x i +m i ·t p

[0027] Among them, T p It's a backdoor image, t p It's a backdoor trigger, x i For a clean image, m i m represents the proportion of the trigger in the image. i ∈[0,1], x is a clean image, y is a clean label, y ′ This is a backdoor label.

[0028] Optionally, the test set consists of images that have been brightened, darkened, and noise-added.

[0029] Optionally, the process of evaluating the trained backdoor model includes:

[0030] The trained backdoor model is evaluated based on the test set, by calculating a clean test set X. c The classification accuracy is used to obtain clean accuracy data; the backdoor test set X, which is misclassified as the target label, is calculated. b X represents all backdoor test sets b The ratio is used to obtain attack success rate data.

[0031] The technical effects of this invention are as follows:

[0032] This invention provides a training method for physical lighting backdoor attacks aimed at artificial intelligence security. The method involves performing a lighting backdoor attack on a target object, further generating a lighting trigger on the target object, and acquiring backdoor image data based on the lighting trigger. Clean image data is then acquired, and a training set is constructed based on the backdoor image data and the clean image data. A backdoor model is constructed, wherein the backdoor model is a deep learning model, and the backdoor model is trained based on the training set to obtain a trained backdoor model. A test set is constructed, and the trained backdoor model is evaluated based on the test set to obtain attack success rate data and clean accuracy data for the lighting backdoor attack.

[0033] The lighting backdoor attack method provided by this invention solves the problems existing in current physical backdoor attacks: it does not require direct access to the target object to deploy the trigger. Furthermore, attackers can proactively launch attacks when needed, making the method of this invention more flexible and covert in initiating backdoor attacks. This invention achieves a more covert physical backdoor attack while maintaining a high attack success rate. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0036] Figure 1 This is a flowchart illustrating the overall process of the image classification experiment in this embodiment of the invention.

[0037] Figure 2 This invention provides a comparison of the concealment of a light-based backdoor attack with existing backdoor attacks.

[0038] Figure 3 Examples of some backdoor images in embodiments of the present invention and Grad-CAM experiments in complex environments;

[0039] Figure 4 This is an example of evaluating the experimental results of other traffic sign test classes in this embodiment of the invention;

[0040] Figure 5 These are experimental results of resisting data augmentation defenses in embodiments of the present invention;

[0041] Figure 6 This is a flowchart illustrating the experimental results of resisting Grad-CAM defense in an embodiment of the present invention. Detailed Implementation

[0042] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as a limitation of the present invention, but rather as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0043] It should be understood that the terminology used in this invention is merely for describing particular embodiments and is not intended to limit the invention. Furthermore, with respect to numerical ranges in this invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Every smaller range between any stated value or intermediate value within a stated range, and any other stated value or intermediate value within said range, is also included in this invention. The upper and lower limits of these smaller ranges may be independently included or excluded from the range.

[0044] Unless otherwise stated, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. While only preferred methods have been described herein, any methods similar or equivalent to those described herein may be used in the implementation or testing of this invention. All references to this specification are incorporated by way of citation to disclose and describe the methods associated with those references. In the event of any conflict with any incorporated reference, the content of this specification shall prevail.

[0045] Various modifications and variations can be made to the specific embodiments described in this specification without departing from the scope or spirit of the invention, as will be apparent to those skilled in the art. Other embodiments derived from this specification will also be obvious to those skilled in the art. This application specification and embodiments are merely exemplary.

[0046] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.

[0047] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0048] Example 1

[0049] like Figures 1-6 As shown, this embodiment provides a training method for physical light backdoor attacks aimed at artificial intelligence security, including:

[0050] A backdoor attack is performed on the target object using light. A corresponding light trigger is generated on the target object based on the light color, and backdoor image data is generated based on the light trigger.

[0051] Obtain clean image data, and construct training sets based on the backdoor image data and the clean image data respectively; the clean image data is the original image without the generation of light triggers;

[0052] Construct a backdoor model, wherein the backdoor model is a deep learning model, and train the backdoor model based on the training set to obtain the trained backdoor model;

[0053] A test set is constructed, and the trained backdoor model is evaluated based on the test set to obtain the attack success rate data and clean accuracy data of the light backdoor attack.

[0054] Use a flashlight to generate light triggers on traffic signs. The light triggers include three different colors: blue, green, and red.

[0055] Use a camera to capture images and obtain the backdoor training set X. b and the backdoor training set X b All images in the target are labeled with the target label.

[0056] Construct training set X p It includes the backdoor training set X b With clean training set X c ,Right now:

[0057] X p =X b ∪X c ,

[0058] The backdoor image can be defined as:

[0059] T p (x i ,t p ,m i )=(1-m i )·x i +m i ·t p ,

[0060] Among them, T p It's a backdoor image, t p It's a backdoor trigger, x i For a clean image, m i m is used to represent the proportion of the trigger in the image. i ∈[0,1].

[0061] Using training set X p Training backdoor model fθ A successfully trained backdoor model should correctly classify clean images but misclassify backdoor images. That is:

[0062] f θ (x) = y, f θ (T p )=y ′

[0063] Where x is a clean image, y is a clean label, and T p For the backdoor image, y ′ This is a backdoor label.

[0064] The success rate and clean accuracy of the light backdoor attack were evaluated using a test set.

[0065] The test set consists of images that have been brightened, darkened, and noise-added (backdoor images and clean images);

[0066] The attacker's capabilities are assumed to include complete control over the training set and the ability to launch an attack using poisoned labels. Furthermore, the attacker only knows the model's architecture but has no control over its internal weights and parameters.

[0067] The attacker's objectives are twofold: 1) The backdoor model correctly classifies clean images. 2) The model incorrectly classifies images with the backdoor.

[0068] The overall flowchart of this embodiment is as follows: Figure 1 As shown, it includes the following three stages: generating backdoor images, training the backdoor model, and evaluating the backdoor model.

[0069] Phase 1: Generating Backdoor Images. A flashlight is used to generate light triggers on traffic signs, with the triggers displaying three different colors: blue, green, and red. A camera is then used to capture and obtain images; these are the "poisoned" images. All backdoor images are then set as target tags.

[0070] Phase 2, Backdoor Model Training. A portion of the backdoor training set was mixed with the clean training set and fed into the model for training. Simultaneously, to verify the impact of the backdoor model on the accuracy on clean data, a clean model was trained using the clean training set (without backdoors) and the same training settings for comparison.

[0071] Phase 3, Backdoor Model Evaluation. This phase evaluates the performance of the trained model. The evaluation consists of two parts: 1) Evaluating the model's classification accuracy on a clean test set; 2) Evaluating the backdoor's attack effectiveness by calculating the percentage of backdoor test sets that were misclassified as the target label out of all backdoor test sets.

[0072] Dataset and Model Structure

[0073] The clean dataset used in this embodiment is the CTSRD traffic sign dataset. Considering the impact of classes with fewer images and low-resolution images on the experimental results, this embodiment selected the 20 classes with the most images and removed images with a resolution of less than 100 pixels in height or width. All images were resized to 224×224×3 pixels. The image classification model architecture used in this embodiment is ResNet-18, ResNet-34, and ResNet-50.

[0074] Experimental setup

[0075] In the image classification experiment of this embodiment, the backdoor attack category is the "No Cars" traffic sign. This embodiment uses the SGD optimizer with a momentum of 0.9, an initial learning rate of 0.01, and 90 epochs. The learning rate decreases by a factor of 10 every 30 epochs. The poisoning rate α is set to 0.02.

[0076] Experimental results

[0077] Results of attack effectiveness experiments:

[0078] Considering the complexity of real-world environments, this experiment performed additional processing on the test set (brightening, darkening, and adding Gaussian noise). This was to simulate changes in brightness in real-world environments and noise that may be introduced during shooting. The evaluation criteria for the effectiveness of the backdoor attack were attack success rate (ASR) and clean data accuracy (CDA). The experimental results are shown in Table 1:

[0079] Table 1

[0080]

[0081] Results of the attack concealment experiment:

[0082] This embodiment is in Figure 2 The stealth of backdoor attacks was compared. It can be seen that images with digital triggers appear suspicious. In contrast, physical backdoor attacks are more stealthy. In the assumed application scenario of this embodiment, the attacker can proactively launch an attack when needed, meaning the traffic signs remain clean for a long period, which is superior to sticker triggers that are always stuck to the traffic signs after deployment. Furthermore, the method in this embodiment is also more stealthy when launching an attack than sticker triggers, which could be mistaken for a roadside light accidentally projecting onto the traffic sign.

[0083] Grad-CAM experimental results:

[0084] Grad-CAM visualizes the prediction process of a DNN using heatmaps, which helps to observe the model's focal areas during the inference phase. For example... Figure 3 As shown, Grad-CAM visualization is used to demonstrate some examples of backdoor images in this embodiment. It can be seen that the backdoor model ignores the original task of the backdoor image and focuses on the backdoor trigger.

[0085] Evaluate the results of other test categories:

[0086] This embodiment uses the previously trained backdoor model to additionally evaluate the backdoor images of the "No Left Turn" and "No Honking" traffic signs.

[0087] This embodiment is in Figure 4 The Grad-CAM visualization shows the focal regions of the three backdoor image classes in the backdoor model and the clean model. It can be seen that the backdoor model primarily focuses on the light trigger, meaning it successfully attacks all three backdoor image classes, while the clean model remains focused on the original image task. This implies that the trained backdoor model is also effective when attacking other classes of backdoor images.

[0088] Backdoor defense experiment results:

[0089] Resisting Data Augmentation: This embodiment verifies whether data augmentation can resist light backdoor attacks by rotating the backdoor image by 30° and randomly cropping it (cropping the image height or width by 30 pixels and reshaping the image to 224×224×3). Figure 5 The image shows the effect of data augmentation on backdoor images, demonstrating that light backdoor attacks are unaffected by data augmentation. The backdoor image is still misclassified because the light triggers are primarily focused on the target object and are not in a static triggering pattern. Therefore, the light triggers are almost unaffected by rotation and random cropping. This means that light backdoor attacks are resistant to data augmentation.

[0090] Defending against Grad-CAM: Grad-CAM heatmaps can be used to capture triggers in backdoor images. While previous Grad-CAM experiments showed this method could capture some backdoor images with lights, this can be circumvented by increasing the area of ​​the light triggers, such as... Figure 6 As shown, when the light trigger almost covers the entire traffic sign, the attention of the back door image in both the clean model and the back door model is mainly focused on the traffic sign, with little difference between the two. The focal area of ​​the back door image in the heatmap of the back door model is only slightly larger, which is not enough to effectively capture the back door.

[0091] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A training method for physical light backdoor attacks aimed at artificial intelligence security, characterized in that, include: A backdoor attack is performed on the target object using light. A corresponding light trigger is generated on the target object based on the light color, and backdoor image data is generated based on the light trigger. Obtain clean image data, and construct training sets based on the backdoor image data and the clean image data respectively; the clean image data is the original image without the generation of light triggers; Construct a backdoor model, wherein the backdoor model is a deep learning model, and train the backdoor model based on the training set to obtain the trained backdoor model; A test set is constructed, and the trained backdoor model is evaluated based on the test set to obtain the attack success rate data and clean accuracy data of the light backdoor attack.

2. The physical light backdoor attack method according to claim 1, characterized in that, The light trigger includes three colors: blue, green, and red.

3. The physical light backdoor attack method according to claim 1, characterized in that, The process of generating the backdoor image data includes: After a light trigger is generated on the target object, the target object from which the light trigger was generated is photographed by a camera to obtain the backdoor image data.

4. The physical light backdoor attack method according to claim 1, characterized in that, The clean image data is a CTSRD traffic sign dataset that has not been subjected to a light backdoor attack.

5. The physical light backdoor attack method according to claim 4, characterized in that, The process of constructing the training set includes: The backdoor image data is used as the backdoor training set X. b and the backdoor training set X b All images in the clean image data are labeled as target labels, and the clean image data is used as the clean training set X. c For the clean training set X c and backdoor training set X b By performing a combined analysis, the training set X is obtained. p ; Wherein, the training set X is obtained. p The calculation formula is: X p =X b ∪X c 。 6. The physical light backdoor attack method according to claim 5, characterized in that, The trained backdoor model f θ for: f θ (x)=y,f θ (T p )=y′ T p (x i ,t p ,m i )=(1-m i )·x i +m i ·t p Among them, T p It's a backdoor image, t p It's a backdoor trigger, x i For a clean image, m i m represents the proportion of the trigger in the image. i ∈[0,1], x is a clean image, y is a clean label, and y′ is a backdoor label.

7. The physical light backdoor attack method according to claim 6, characterized in that, The test set consists of images that have been brightened, darkened, and noise-added.

8. The physical light backdoor attack method according to claim 7, characterized in that, The process of evaluating the trained backdoor model includes: The trained backdoor model is evaluated based on the test set, by calculating a clean test set X. c The classification accuracy is used to obtain clean accuracy data; the backdoor test set X, which is misclassified as the target label, is calculated. b X represents all backdoor test sets b The ratio is used to obtain attack success rate data.

Images