Backdoor classification detection method based on pre-training model amplification strategy

Abstract

The invention relates to a back door classification detection method based on a pre-training model amplification strategy, and is suitable for the field of attack and defense. According to the technical scheme, the method comprises the steps that a pre-training classification model with a back door serves as an original model, parameters of a BN layer are gradually and simultaneously amplified in the forward direction from the kth layer to the (k + n-1) th layer of the original model, and n parameter amplification models are obtained; inputting a to-be-detected image into the original model and the n parameter amplification models, and obtaining a prediction label of the original model and prediction confidence of the n parameter amplification models on the prediction label of the original model; and based on the prediction confidence of the n parameter amplification models, judging whether the to-be-detected image is a backdoor sample, and if the to-be-detected image is judged to be the backdoor sample, judging the contained backdoor based on the prediction label of the original model.

Description

Backdoor Classification and Detection Method Based on Pre-trained Model Amplification Strategy Technical Field

[0001] This invention relates to a backdoor classification and detection method based on a pre-trained model amplification strategy. It is applicable to adversarial attack and defense fields. Background Technology

[0002] With the rapid development of neural networks, researchers are constantly increasing the number of layers and the size of parameters in order to improve the performance and accuracy of their models. However, while more layers and more neurons can bring better accuracy, they also introduce some problems.

[0003] One of the more common problems is backdoor attacks. Unlike adversarial attacks, backdoor attacks are attacks that occur during the training phase. Attackers insert backdoor triggers into the target model before or during training, allowing them to precisely control the model's predictions during the testing phase. With the rise of Machine Learning as a Service (MAS) and Model as a Service (MAS), and the increasing reliance on network data for training large models, backdoor attacks have become the second largest model security threat after adversarial attacks.

[0004] The objectives of a backdoor attack are: (1) the backdoor model has normal accuracy on clean test samples; and (2) the backdoor model will produce a prediction result pre-specified by the attacker if and only if the test samples contain a pre-set backdoor trigger. Among these objectives, objective (1) ensures the stealth of the backdoor attack, and objective (2) ensures that the backdoor model can be manipulated arbitrarily by the attacker.

[0005] Backdoor attacks are accomplished through two operations: backdoor implantation and backdoor activation. Backdoor implantation refers to the attacker inserting a predefined backdoor trigger into the target model during the training phase, thereby obtaining a backdoor model. Backdoor activation refers to the fact that during the inference phase, any test sample containing the backdoor trigger will activate the backdoor and control the model to output the prediction result specified by the attacker.

[0006] Backdoor attacks are often characterized by low attack threshold, high success rate, and high stealth. On one hand, this is because once a backdoor trigger is injected into the target model, it can easily be used to launch an attack. On the other hand, the backdoor model behaves normally on clean samples, and only when the backdoor trigger appears will the model exhibit malicious behavior, making backdoor attacks very difficult to detect through ordinary model testing.

[0007] Backdoor attacks are generally considered a special type of data poisoning attack, although their implementation is not limited to data poisoning (it can also involve directly modifying model parameters). Traditional data poisoning attacks aim to reduce the model's generalization performance, while backdoor attacks aim to control the model's prediction results through backdoor triggers. In other words, backdoor attacks are targeted, manipulative attacks; their goal is to control the model's output to a specific category that benefits the attacker.

[0008] Given that training deep neural networks often requires a large amount of training data and places high demands on computing resources, ordinary users typically cannot meet these requirements simultaneously. Therefore, some model developers may choose to outsource training tasks to third-party platforms or fine-tune downstream tasks directly on publicly available pre-trained models. In this situation, attackers may implant backdoors into the model during the training process on the third-party platform; simultaneously, malicious attackers may also upload models containing backdoors to public platforms, such as GitHub, for victims to download and use.

[0009] Existing backdoor classification and detection technologies are mainly designed for detecting single backdoor attack methods or multiple models for detecting various backdoor attack methods, but they are difficult to detect new backdoor attack methods and are costly. Summary of the Invention

[0010] The technical problem to be solved by this invention is to provide a backdoor classification and detection method based on a pre-trained model amplification strategy, which addresses the problems mentioned above.

[0011] The technical solution adopted in this invention is: a backdoor classification and detection method based on a pre-trained model amplification strategy, comprising: using a pre-trained classification model with a backdoor as the original model, and gradually and simultaneously amplifying the parameters of the BN layer from the kth layer to the k+n-1th layer from the end of the original model to obtain n parameter amplification models; inputting the image to be detected into the original model and the n parameter amplification models to obtain the predicted label of the original model and the prediction confidence of the n parameter amplification models on the predicted label of the original model; based on the prediction confidence of the n parameter amplification models, determining whether the image to be detected is a backdoor sample; if it is determined to be a backdoor sample, then determining the backdoor it contains based on the prediction label of the original model.

[0012] The pre-trained classification model with backdoors includes: generating backdoor data corresponding to each backdoor attack algorithm based on image data using at least one backdoor attack algorithm; combining the image data with the backdoor data corresponding to each backdoor attack algorithm and assigning backdoor attack labels to the backdoor data to form training datasets corresponding to each backdoor attack algorithm; and training the pre-trained classification model sequentially based on the training datasets corresponding to each backdoor attack algorithm until the proportion of backdoor data that successfully attack the target label is greater than a preset value, at which point the backdoor addition is completed, and the pre-trained classification model with backdoors is obtained.

[0013] The method of determining whether the image to be detected is a backdoor sample based on the prediction confidence of the amplification model with n parameters includes: calculating the average value of the prediction confidence based on the prediction confidence of the amplification model with n parameters, comparing whether the average value is greater than a preset value, and if it is greater than the preset value, then determining that the image to be detected is a backdoor sample.

[0014] A backdoor classification and detection device based on a pre-trained model amplification strategy includes: a parameter amplification module, used to simultaneously and progressively amplify the parameters of the BN layer from the k-th to k+n-1 layers from the end of the original model, using a pre-trained classification model with a backdoor as the original model, to obtain n parameter amplification models; a model prediction module, used to input the image to be detected into the original model and the n parameter amplification models, to obtain the predicted label of the original model and the prediction confidence of the n parameter amplification models on the predicted label of the original model; and a backdoor judgment model, used to determine whether the image to be detected is a backdoor sample based on the prediction confidence of the n parameter amplification models, and if it is determined to be a backdoor sample, to determine the backdoor it contains based on the prediction label of the original model.

[0015] A storage medium storing a computer program executable by a processor, wherein the computer program, when executed, implements the steps of the backdoor classification and detection method based on a pre-trained model amplification strategy.

[0016] A backdoor classification and detection device includes a memory and a processor. The memory stores a computer program that can be executed by the processor. When the computer program is executed, it implements the steps of the backdoor classification and detection method based on the pre-trained model amplification strategy.

[0017] The beneficial effects of this invention are: This invention amplifies a certain sample to be predicted pixel by pixel. The prediction result of backdoor samples is stable, while the prediction result of normal samples will gradually change. Based on the original model, the parameters of the BN layer are amplified to obtain an n-parameter amplified model, and the prediction confidence of the n-parameter amplified model on the original model prediction label is used to determine whether the image to be detected is a backdoor sample. Attached Figure Description

[0018] Figure 1 is a flowchart of an embodiment.

[0019] Figure 2 is a schematic diagram showing the enlarged model parameters in the embodiment.

[0020] Figure 3 is a schematic diagram of the backdoor sample judgment in the embodiment, where the left side is judged as a normal sample and the right side is judged as a backdoor sample. Detailed Implementation

[0021] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0022] In the description of this invention, "multiple" means two or more. The use of "first" and "second" is for distinguishing technical features only and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or the order of the indicated technical features. Furthermore, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art.

[0023] Example 1: This example is a backdoor classification and detection method based on a pre-trained model amplification strategy, which specifically includes the following steps: S100, based on image data, using at least one backdoor attack algorithm, generating backdoor data corresponding to each backdoor attack algorithm.

[0024] Considering the variety of backdoor attack algorithms and the emergence of new backdoor attack algorithms in the future, this embodiment develops an extensible backdoor attack algorithm module. Users can upload their backdoor attack algorithms according to the interface settings to extend the system and achieve the effect of expanding backdoor detection.

[0025] In this embodiment, four backdoor attack algorithms are selected as the basic attack algorithms, which are representative methods in two categories of dirty label attack backdoor attacks: 1) Input space attack: Badnet, blend; 2) Model space attack: Trojan, SRA.

[0026] BadNet attack. Gu et al. first investigated potential backdoor vulnerabilities in current machine learning paradigms and training processes, proposing the BadNets attack algorithm to insert backdoors into deep learning models during training. BadNets is a classic dirty labeling attack method that explores two threat scenarios: training task outsourcing and pre-trained models. It's worth noting that backdoor attacks based on data poisoning generally assume that attackers can only inject a small number of backdoor samples into the training data but cannot control the model's training process. BadNets' threat model is relatively more lenient; attackers can both access the training data and control the model's training.

[0027] Blend Attack. Chen et al. proposed the Blend attack, a novel trigger design and improvement upon the BadNets attack. The Blend attack uses two novel triggers: global random noise and an image blending strategy. This attack allows the backdoor trigger to no longer be limited to a specific region of the image (in the BadNets attack, the trigger is fixed in the lower right corner of the image). Simply put, the global random noise-based attack uses random noise as a backdoor trigger, superimposing it onto a clean sample, while the image blending-based attack uses a specified image as a backdoor trigger, superimposing it onto a clean sample.

[0028] Trojan attacks. The Trojan attack, proposed by Liu et al., is the first model space backdoor attack method. The threat model of a Trojan attack closely resembles reality because, in actual application scenarios, key processes such as data collection and model training are often controlled by model vendors. These processes require significant resources, so attackers have no incentive to incur high costs to insert a backdoor. However, Trojan attacks allow attackers to directly attack pre-trained models, greatly reducing the cost of the attack. Simply put, the goal of a Trojan attack is to attack a pre-trained model when the training data is unknown and unavailable. The Trojan attack process generally consists of three steps: Trojan pattern generation, training data generation, and Trojan implantation.

[0029] SRA Attack. Xiangyu Qi et al. investigated the realistic potential of backdoor attacks on deep neural networks (DNNs) during the deployment phase. Specifically, the goal of their approach was to design a deployment-phase backdoor attack algorithm that is both threatening and practically achievable. To this end, they proposed the Subnet Replacement Attack (SRA), which can embed a backdoor into a DNN by directly modifying a finite number of model parameters. Considering practical applicability, they abandoned the strong white-box assumption widely adopted in existing research; instead, their algorithm operates in a gray-box setting, where the architectural information of the victim model is available, but the adversary is unaware of the parameter values.

[0030] The four categories are related yet distinct to each other, which effectively demonstrates the model's effectiveness and the relative independence between backdoor attacks.

[0031] S200. Combine the image data with the backdoor data corresponding to each backdoor attack algorithm, and assign the backdoor data with a backdoor attack label to form the training dataset corresponding to each backdoor attack algorithm.

[0032] In this example, the backdoor data is separated from the original image data, rather than being directly combined with the original data to form a new dataset. This significantly reduces disk space consumption and facilitates integration while maintaining the relative independence of each backdoor. During data loading, the dataset is reconstructed, backdoor data is added, and labels for each backdoor attack are assigned, forming the training dataset corresponding to each backdoor attack algorithm.

[0033] S300. Based on the training datasets corresponding to each backdoor attack algorithm, train the pre-trained classification model in sequence until the proportion of backdoor data that successfully attack the target label is greater than the preset value. Then the backdoor addition is completed, and a pre-trained classification model with backdoor is obtained.

[0034] In this embodiment, the classification model is trained using a strategy of iteratively training a single model. The model is trained sequentially based on the training dataset corresponding to each backdoor attack algorithm. When the ASR (the proportion of backdoor data that successfully attacks the target label out of all backdoor data) is greater than 0.9, it is considered that the current backdoor has been successfully added. Then, new backdoor data is replaced and the data loading process continues. This training process is repeated until all backdoors have been added.

[0035] S400. Using the pre-trained classification model with a backdoor as the original model, starting from the kth layer to the k+n-1th layer from the end of the original model, the parameters of the BN layer are gradually and simultaneously amplified in the positive direction to obtain a model with n amplified parameters.

[0036] Batch Normalization (BN) layers transform the mean and variance of the original data into the learnable values ​​β and γ. When the parameters β and γ of the BN layer are increased, the dispersion of the normal distribution increases, leading to a decrease in the prediction confidence of normal samples. However, backdoor samples will still exhibit higher prediction confidence.

[0037] Based on the above characteristics, this embodiment starts from the kth layer to the k+n-1th layer from the end of the original model and gradually and simultaneously amplifies the parameters of the BN layer in the positive direction (here, ω=1.5 times amplification) to obtain n different parameter amplification models.

[0038] In this embodiment, k and n are both positive integers and can be adjusted according to the classification model architecture. However, experiments have shown that if k and n are large, it will have the opposite effect. In this example, the ResNet18 architecture is used, with k=7 and n=5.

[0039] S500: Input the image to be detected into the backdoor detection model (which includes the original model and n-parameter amplification models), obtain the predicted label of the original model, and the prediction confidence of the n-parameter amplification models on the predicted label of the original model.

[0040] S600. Based on the prediction confidence of the amplified model with n parameters on the original model's predicted label, calculate the average of the n prediction confidences, and compare whether the average is greater than a preset value T (here set to 0.9) to determine whether it is a backdoor sample (poisoned samples with added trigger patterns are "backdoor samples"). If it is determined to be a backdoor sample, then based on the label predicted by the original model, combined with the backdoor data corresponding to each backdoor attack algorithm in step S200, and the backdoor attack label assigned to the backdoor data, determine the backdoor it contains.

[0041] When normal samples pass through the BN layer, the data distribution follows a normal distribution. However, backdoor samples exhibit extreme behavior, essentially exhibiting a 0-1 distribution. They are 1 for the target of the backdoor attack and 0 for others. Therefore, after amplifying the parameters of the BN layer during inference, the overall confidence of normal samples will decrease, but the target of the backdoor attack will not be affected. This characteristic can be used to capture backdoor samples.

[0042] Example 2: This example is a backdoor classification and detection device based on a pre-trained model amplification strategy. Specifically, it includes: a parameter amplification module, which uses a pre-trained classification model with a backdoor as the original model, and gradually amplifies the parameters of the BN layer simultaneously from the kth layer to the k+n-1th layer from the end of the original model to obtain n parameter amplification models; a model prediction module, which inputs the image to be detected into the original model and the n parameter amplification models, obtains the predicted label of the original model, and the prediction confidence of the n parameter amplification models on the predicted label of the original model; and a backdoor judgment model, which determines whether the image to be detected is a backdoor sample based on the prediction confidence of the n parameter amplification models. If it is determined to be a backdoor sample, it determines the backdoor it contains based on the prediction label of the original model.

[0043] Example 3: This example is a storage medium that stores a computer program that can be executed by a processor. When the computer program is executed, it implements the steps of the backdoor classification and detection method based on the pre-trained model amplification strategy described in Example 1.

[0044] Example 4: This example is a backdoor classification and detection device, which has a memory and a processor. The memory stores a computer program that can be executed by the processor. When the computer program is executed, it implements the steps of the backdoor classification and detection method based on the pre-trained model amplification strategy described in Example 1.

[0045] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the aforementioned functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0046] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0047] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0048] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0049] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0050] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0051] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0052] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A backdoor classification and detection method based on a pre-trained model amplification strategy, characterized in that, include: Using a pre-trained classification model with a backdoor as the original model, the parameters of the BN layer are gradually and simultaneously amplified from the kth layer to the k+n-1th layer from the end of the original model, resulting in an n-parameter amplified model. The image to be detected is input into the original model and the n-parameter amplified model to obtain the predicted label of the original model and the prediction confidence of the n-parameter amplified model on the predicted label of the original model. Based on the prediction confidence of the n-parameter amplified model, it is determined whether the image to be detected is a backdoor sample. If it is determined to be a backdoor sample, the backdoor contained in it is determined based on the prediction label of the original model.

2. The backdoor classification and detection method based on a pre-trained model amplification strategy according to claim 1, characterized in that, The pre-trained classification model with backdoors includes: generating backdoor data corresponding to each backdoor attack algorithm based on image data using at least one backdoor attack algorithm; combining the image data with the backdoor data corresponding to each backdoor attack algorithm and assigning backdoor attack labels to the backdoor data to form training datasets corresponding to each backdoor attack algorithm; and training the pre-trained classification model sequentially based on the training datasets corresponding to each backdoor attack algorithm until the proportion of backdoor data that successfully attack the target label is greater than a preset value, at which point the backdoor addition is completed, and the pre-trained classification model with backdoors is obtained.

3. The backdoor classification and detection method based on a pre-trained model amplification strategy according to claim 1, characterized in that, The method of determining whether the image to be detected is a backdoor sample based on the prediction confidence of the amplification model with n parameters includes: calculating the average value of the prediction confidence based on the prediction confidence of the amplification model with n parameters, comparing whether the average value is greater than a preset value, and if it is greater than the preset value, then determining that the image to be detected is a backdoor sample.

4. A backdoor classification and detection device based on a pre-trained model amplification strategy, characterized in that, include: The parameter amplification module is used to take the pre-trained classification model with backdoors as the original model and gradually amplify the parameters of the BN layer simultaneously from the kth layer to the k+n-1th layer from the end of the original model to obtain n parameter amplification models. The model prediction module is used to input the image to be detected into the original model and the n parameter amplification models to obtain the predicted label of the original model and the prediction confidence of the n parameter amplification models on the predicted label of the original model. The backdoor detection model is used to determine whether the image to be detected is a backdoor sample based on the prediction confidence of the n parameter amplification models. If it is determined to be a backdoor sample, it determines the backdoor it contains based on the prediction label of the original model.

5. A storage medium storing a computer program executable by a processor, characterized in that: When the computer program is executed, it implements the steps of the backdoor classification and detection method based on the pre-trained model amplification strategy as described in any one of claims 1 to 3.

6. A backdoor classification and detection device, comprising a memory and a processor, wherein the memory stores a computer program executable by the processor, characterized in that: When the computer program is executed, it implements the steps of the backdoor classification and detection method based on the pre-trained model amplification strategy as described in any one of claims 1 to 3.

Images