Neural network-oriented computable hidden backdoor attack method
The improved backdoor attack method disperses trigger functions into invisible disturbances using projected gradient descent, enhancing attack efficacy and concealment in neural networks, effectively bypassing detection.
Patent Information
- Application Number
- CN202411120003.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-12-03
AI Technical Summary
Existing backdoor attack methods in transfer learning scenarios for neural networks are ineffective in altering the classification ability of models and fail to ensure trigger concealment, making them easily detectable.
An improved computable covert backdoor attack method using projected gradient descent to update triggers and perturbations, dispersing the trigger's function into an invisible disturbance, ensuring the attack effect while evading human and algorithmic detection.
The method enhances the attack's effectiveness by increasing trigger discovery probability and improving concealment, allowing the attack to bypass defense mechanisms and evade human visual recognition.
Smart Images

Figure CN119066653A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep learning security, and in particular to a backdoor attack method for a neural network. Background Art
[0002] Deep Neural Networks (DNNs) have been widely used in target detection, natural language processing, and smart healthcare. Its success is mainly due to the huge model parameters and massive training data, but this is extremely difficult for some resource-constrained users because it requires a lot of manpower and material resources. So people began to obtain computing power and training data from third-party platforms, such as Github and Hugging face.
[0003] However, deep learning models are highly sensitive to training data. If an attacker impersonates a third-party platform and adds some poisoned data to the training data, a backdoor may be implanted in the model. A backdoor attack will make the model perform well on clean data, but for poisoned data containing triggers, it will cause the model to output a malicious result pre-specified by the attacker. The essence of a backdoor attack is to use the redundant classification capabilities of the neural network model to train the model to learn the potential relationship between the trigger and the target category. Although there has been a lot of work related to backdoor attacks, for scenarios where only a small number of parameters can be updated - for example, transfer learning usually only updates the weights of the last layer of the model, the existing attack methods are not sufficient to affect the classification ability of the model itself, resulting in the ineffectiveness of the attack. Or although the attack can be implemented, the concealment of the trigger cannot be guaranteed, making the attack easy to be discovered.
[0004] Projected Gradient Descent (PDG) is an optimization algorithm mainly used to solve optimization problems with constraints, especially to deal with regularization problems in the field of machine learning. PDG combines the gradient descent method and the projection operation to iteratively update the parameters to minimize the objective function. In each iteration, PDG first updates in the direction of the negative gradient, which is the same as the standard gradient descent method. However, due to the existence of constraints, simple gradient descent may cause the parameters to exceed the allowed range. Therefore, after the update, PDG performs a projection operation to project the updated parameters back into the feasible domain that satisfies the constraints. This process ensures that after each iteration, the parameters are still within the constraint set, thereby maintaining the feasibility of the problem. The present invention uses this method to ensure that the order of magnitude of the disturbance is limited to a certain range. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide an improved backdoor attack method which can be used in a transfer learning scenario to improve the attack effect and concealment and can effectively bypass defense model detection.
[0006] The technical solution adopted by the present invention to solve the technical problem is: an improved computable hidden backdoor attack method, comprising the following steps:
[0007] S1, copy all weights of the neural network from the first layer to the second to last layer, and initialize the input data including the trigger;
[0008] S2. Input the initialization data obtained in step S1 into the model and calculate the activation value output by the model. Traverse the activation value of each neuron, define the mean square error between this activation value and the target activation value as the loss function, and update the value of the trigger;
[0009] S3. Initialize a small perturbation, extract a certain number of samples from the training data, divide them into two categories and input them into the model, calculate the activation values of the two categories of samples in the penultimate layer, define their mean square error as the loss function, and update the value of the perturbation;
[0010] S4, inserting the obtained perturbed data into normal training data to train the neural network, and obtaining a neural network model with a hidden backdoor implanted;
[0011] S5. Determine whether the attack effect meets the requirements. If so, upload the entire training set to the open source platform according to the feedback information and wait for the victim to download it. Otherwise, adjust the trigger parameters and hiding strategy according to the feedback information and return to step S2.
[0012] Specifically, the input data in S1 is a 3×224×224 pixel image, and the trigger is a 3×30×30 pixel square. They are combined by formula (1), where M represents a mask that has the same size as the input data, and the position of the trigger is adjusted by adjusting the position of 1 in M.
[0013] T(x)=(1-M)⊙x+M⊙t (1)
[0014] Specifically, the gradient descent method is used in S2 to update the value of the trigger. This method can ensure that the loss is gradually reduced, so that the activation value of the neuron gradually approaches the target value. Its formal definition is shown in formula (2). In addition, step S2 only records the activation value change of the penultimate layer of the model, aiming to find a neuron with the largest activation value change and the corresponding trigger. On the basis of ensuring the original function of the model, the trigger is as easy to be perceived by the neural network model as possible.
[0015]
[0016]
[0017] Specifically, the method used to update the perturbation in S3 is the projected gradient descent method. It is mainly used to solve optimization problems with constraints. In each iteration, PDG first updates in the direction of the negative gradient, which is the same as the standard gradient descent method. This may cause the parameters to exceed the allowed range. Therefore, after the update, PDG will perform a projection operation to project the updated parameters back to the feasible domain that satisfies the constraints, ensuring that the difference between the image after adding the perturbation and the original image at the pixel level is small. Its formal definition is shown in formula (3), where x represents clean data, t * represents a computable trigger, p represents the poisoned data with disturbance, Indicates poison data that adds computable triggers.
[0018]
[0019] st.∣∣px∣∣ ∞ <∈
[0020] The beneficial effects of the present invention are as follows:
[0021] The computable covert backdoor attack described in the present invention can increase the probability of triggers being discovered by neural networks in terms of trigger generation, thereby improving the effect of the attack; at the same time, it can disperse the effect of the trigger onto an invisible disturbance, hide the trigger, and easily evade the human visual recognition system. When faced with thermal map analysis, there is no obvious difference between the poisoned data and the clean data of the present invention; when faced with defense detection methods, the poisoned data of the present invention can successfully bypass the detection mechanism and is not discovered. The computable covert backdoor attack method described in the present invention can not only improve the effect of the attack, but also improve the concealment of the attack when facing human and algorithm detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 A schematic diagram of a backdoor attack application scenario of the present invention;
[0023] Figure 2 It is a flow chart of the computable hidden backdoor attack method for neural networks of the present invention;
[0024] Figure 3 A schematic diagram of a computable trigger generation process of the present invention;
[0025] Figure 4 It is a schematic diagram of the invisible disturbance generation process of the present invention;
[0026] Figure 5Schematic diagram of the highly efficient hidden poisoning data generated by the present invention, where ∈ represents the order of magnitude of the disturbance, and the smaller the value, the more hidden it is. The present invention can reduce ∈ to 2;
[0027] Figure 6 This is a comparative schematic diagram of bypassing defense detection in the present invention. DETAILED DESCRIPTION
[0028] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0029] Figure 1 It is a schematic diagram of the backdoor attack application scenario of the present invention. With the continuous increase in the scale of model parameters and the scale of training data, the use of third-party pre-trained models and open source data sets has become more and more common for fine-tuning pre-trained models. In this process, although the user can control the fine-tuning process and model structure by himself, it is impossible to conduct a comprehensive inspection of each open source data, which provides the possibility for the backdoor attack of the present invention. The attacker uses the method of the present invention to upload poisoned data to the open source data, so that the model suitable for downstream tasks trained based on these data is implanted with a backdoor. This is an extremely common scenario, so the attack of the present invention is of great significance to the security research of deep neural networks.
[0030] Figure 2 It is a flowchart of the computable hidden backdoor attack method for neural networks of the present invention. Figure 1 The attacker's execution process is refined. It mainly includes the following steps: S1, initialization of the model and trigger, S2, generation of computable triggers, S3, generation of invisible triggers, S4, model training and implantation of backdoors. In the model initialization stage, the attacker does not need to obtain all the weights of the model, but only needs to obtain the weights from the first layer to the penultimate layer. During the trigger initialization process, it is necessary to determine the size, position and shape of the trigger, and randomly initialize the pixel value. The present invention makes the trigger into a square of 3×30×30 pixels. In order to ensure that the trigger can be input into the model, the trigger will be embedded into an image through formula (1) as input data.
[0031] After initialization, the input data is sent to the model for training, with the goal of generating a computable trigger with a stronger attack effect. The core idea is to increase the probability of the trigger being discovered by the model, making it easier for the model to learn the potential connection between the trigger and the attack category, thereby ensuring the efficiency of the backdoor attack. However, such a trigger is not concealed and can be directly discovered by the human visual recognition system. Therefore, the present invention disperses the function of the computable trigger into an invisible disturbance. This makes the poisoned data indistinguishable from the clean data both to the naked eye and under the defense detection algorithm, ensuring the concealment of the backdoor attack.
[0032] Finally, inserting these poisoned samples into the training data will result in a model with a backdoor implanted. During the inference process, when the input sample is attached with a computable trigger, the hidden backdoor in the model will be activated, resulting in the output of malicious results specified by the attacker. For normal clean samples, the model will perform normally.
[0033] Figure 3 is a schematic diagram of the computable trigger generation process of the present invention, which is Figure 2 Refinement of the flowchart of the backdoor attack method. The computable trigger needs to find a "potential" neuron - it will undergo a significant change in activation value due to the presence of the trigger, making it easier for the model to detect the presence of the trigger. The present invention proposes an improved neuron search method, which iteratively calculates the mean square error between the activation value output by each neuron and the target activation value as the loss function, and uses the gradient descent algorithm to update the value of the trigger so that the value of the loss function gradually decreases. At this time, the activation value of the neuron will gradually increase and approach the target activation value. Its formal definition is shown in formula (2), and the specific process is shown in algorithm 1.
[0034]
[0035] Figure 4 is a schematic diagram of the invisible disturbance generation process of the present invention, which is also Figure 2 Refinement of the flowchart of the backdoor attack method. In order to disperse the function of the trigger into an invisible perturbation, the present invention calculates the activation values A1 and A2 of the two types of poisoned data, and defines their mean square error as the loss function, loss = MSE (A1, A2). As shown in formula (3), the invisible perturbation is updated using the projected gradient descent algorithm, which can ensure that the function of the computable trigger is dispersed into the invisible perturbation, and can also ensure that the infinite norm of the subtraction between the perturbed image and the original image is controlled within a certain range.
[0036] Table 1 shows the test results of different attack methods on different data sets. The test indicators include accuracy (ACC) and attack success rate (ASR), which are defined as shown in formula (4) and formula (5). The attack effect of the present invention has achieved the best results on each data set.
[0037] Table 1
[0038]
[0039] Among them, TP (True Positives) represents the number of samples correctly predicted as positive; TN (True Negatives) represents the number of samples correctly predicted as negative; FP (False Positives) represents the number of samples incorrectly predicted as positive; FN (False Negatives) represents the number of samples incorrectly predicted as negative; card (D poison ) represents the poisoned sample set D poison The number of elements in .
[0040]
[0041] Figure 5 : This is a schematic diagram of the highly efficient hidden poisoned data generated by the present invention, where ∈ (epsilon) represents the order of magnitude of the disturbance. ∈ directly affects the concealment of the poisoned data, thereby affecting the possibility of being discovered by the user. It can be found that when ∈ = 32, the disturbance in the poisoned data can be clearly seen. As ∈ continues to decrease, the poisoned image gradually approaches the original image. The backdoor attack method of the present invention can reduce ∈ to 2 without affecting the attack performance.
[0042] Figure 6 This is a comparative diagram of the present invention bypassing defense detection. For a backdoor attack method, in addition to being able to deceive the human visual recognition system, it is also necessary to be able to bypass the existing backdoor defense detection system. The present invention tested the performance of poisoned data under the Neural Clean detection system and found that the poisoned data discovered in this invention did not exceed the threshold set by the detection system (the value is 2), and therefore successfully bypassed the defense.
[0043] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A computable hidden backdoor attack method for neural networks, characterized in that: The method comprises the following steps: S1, copy all weights of the neural network from the first layer to the second to last layer, and initialize the input data including the trigger; S2. Input the initialization data obtained in step S1 into the model and calculate the activation value output by the model. Traverse the activation value of each neuron, use the mean square error between the activation value and the target activation value as the loss function, and update the value of the trigger; S3. Initialize a small perturbation, extract a certain number of samples from the training data, divide them into two categories and input them into the model, calculate the activation values of the two categories of samples in the penultimate layer, define their mean square error as the loss function, and update the value of the perturbation; S4, inserting the perturbed data into the normal training data, training the neural network, and obtaining a neural network model with a hidden backdoor implanted; S5. Determine whether the attack effect meets expectations. If so, upload the entire training set to the open source platform based on the feedback information for the victim to download. If not, adjust the trigger parameters and hiding strategy based on the feedback information and return to step S2.
2. The backdoor attack method according to claim 1, characterized in that: The trigger is a square with a height and width of 30 pixels, located in the right corner of the data, 5 pixels away from the right border and 5 pixels away from the bottom border. In the initialized input data, the data at the trigger position is a random value, and the values of the other positions are all 0.
3. The backdoor attack method according to claim 1, characterized in that: Step S2 uses the gradient descent method to update the trigger value, and only records the activation value changes of the penultimate layer of the model, looking for the neuron with the largest activation value change and its corresponding trigger. On the basis of ensuring the original function of the model, the possibility of the trigger being detected by the model is increased as much as possible.
4. The backdoor attack method according to claim 1, characterized in that: Step S3 includes the following sub-steps: S301. Use the stratified random sampling method to extract the same number of samples from each category of the training data. S302, initialize a square with a height and width of 224 pixels as a disturbance, and its pixel values conform to a Gaussian distribution with a mean of 0 and a variance of 1. S303, classify the extracted samples according to "whether they are attack target categories", paste the trigger obtained in step S2 for samples of non-attack target categories, and paste the disturbance obtained in step S402 for samples of target categories. S304. Input the two types of samples into the model to calculate the activation value of the penultimate layer of the model, calculate the loss according to the defined loss function, use the projected gradient descent algorithm to update the perturbation to reduce the loss, diffuse the function of the trigger into the perturbation, and ensure that there is no obvious visual difference between the perturbed image and the original image. S305: Determine whether the loss meets the iteration end condition. If so, the iteration ends; otherwise, return to step S304.
5. The backdoor attack method according to claim 1, characterized in that: The trigger of the poisoned data is an invisible perturbation, and the label of the poisoned data has not changed. For the formal training process of the neural network, the attacker cannot and does not need to control it. When testing the attack success rate of the neural network implanted with the backdoor, the trigger used is the computable trigger obtained in step S2, not the invisible perturbation obtained in step S3.
6. The backdoor attack method according to claim 1, characterized in that: According to the feedback information, the training parameters are adjusted, including the training learning rate, the target activation value of neurons, the number of training times, etc. Or the poisoned data is inserted into the training data, uploaded to an open source data platform, and waited for the victim to download it. In this way, it is possible to implant an efficient and hidden backdoor into the neural network model without controlling the training process.
Citation Information
Cited By
Federal learning-oriented anti-backdoor attack image segmentation model training method and device
CN120563969A
