Image recognition backdoor defense method, device and medium based on swarm intelligence

Abstract

The present invention discloses a method, device and medium for backdoor defense of image recognition based on swarm intelligence. Based on the association relationship of multiple neural network image recognition models in a distributed cluster and ranking according to the degree of connection density, a densely connected neural network model is selected as a seed neural network model; for each two seed neural network models, an attention distillation representation is used to perform a distillation operation on the two seed neural network models to extract an attention activation map that is invalid for backdoor attacks on the seed neural network models. A l ; By designing a robust distillation loss function L Measuring the activation attention maps of two seed neural network models A l The Euclidean distance and cosine distance of the seed neural network model are based on L Calculate the gradient value and perform back propagation to update the parameters of the seed neural network model. The performance and efficiency of the swarm intelligence image recognition backdoor defense algorithm proposed in the present invention under the image recognition backdoor defense task have reached the current optimal level.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to an image recognition backdoor defense method, device, product and storage medium based on swarm intelligence. Background Art

[0002] The application of neural network image recognition in autonomous driving and medical diagnosis has developed rapidly. However, since most neural network image recognition is optimized in a performance-driven manner, not much attention is paid to whether the neural network will be attacked and tampered by malicious people. People are increasingly concerned about the security of neural networks in image recognition applications. Due to the opacity of neural networks, adversarial attacks and backdoor attacks have become the main factors threatening the security of neural networks. Adversarial attacks mislead the model into predicting errors by carefully crafting adversarial samples. However, crafting strong adversarial training samples is computationally expensive because it requires iterative gradient steps relative to the loss function. [1] However, compared with adversarial attacks, the cost of implanting backdoor attacks in image recognition tasks is lower. It only requires data poisoning to implant backdoors in image recognition models. In addition, backdoor attacks have little impact on the accuracy of image recognition models and are difficult to detect when triggered. This attack mode is more favored by malicious attackers in practical applications.

[0003] Although there are many methods for detecting backdoor attacks in neural network image recognition, these methods only perform detection on a single neural network backdoor attack. With the surge in computing power, most servers for training neural networks use distributed clusters to provide computing resources. Performing detection on backdoor attacks on a single neural network not only leads to very low erasure efficiency, but also makes it very easy to cross-implant backdoor triggers in the distributed cluster, reducing the erasure capability. Secondly, there is no God's eye in the real world that can clearly know whether an image recognition model has a backdoor attack before the algorithm is deployed. Some previous work [2] First, a model is trained by using a subset of image data without backdoor triggers, and then the model containing backdoor attacks is fine-tuned or distilled based on the image recognition model without backdoor triggers to improve the robustness against backdoor attacks. This robustness can be transferred between different tasks, but it requires a model that does not contain backdoor attacks, which is a big limitation.

[0004] In summary, the prior art has the following defects:

[0005] (1) Existing image recognition backdoor defense methods have not explored and studied the defense efficiency of backdoor attacks, which makes it difficult to perform efficient defense in a distributed cluster environment in a real environment.

[0006] (2) In traditional image recognition neural networks, image recognition models without backdoor triggers are trained one by one through image data without backdoor triggers implanted in order to erase the triggers of backdoor attacks, which greatly limits the erasure efficiency. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide an image recognition backdoor defense method, device, product and storage medium based on swarm intelligence to address the deficiencies in the prior art, thereby significantly improving the defense efficiency of the neural network model cluster against backdoor attacks.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is: an image recognition backdoor defense method based on swarm intelligence, comprising the following steps:

[0009] S1. Based on the association relationship of multiple neural network image recognition models in the distributed cluster, the neural network model with dense ranking connections is selected as the seed neural network model (for example, a server is connected to 5 servers in the entire distributed cluster, which is counted as 5. All servers in the distributed cluster are sorted according to the value of the connection, and the top 10% are selected as the seed neural network model); then, for each two seed neural network models, the attention distillation representation is used to perform a distillation operation on the two seed neural network models to extract the attention activation map A that is invalid for the backdoor attack of the seed neural network model l ;

[0010] S2, by using the channel summation function The attention activation maps of multiple channels are summed up to obtain

[0011] S3. Multi-channel summation using activated attention maps The Euclidean distance and cosine distance of the activation attention maps of the two seed neural network models are measured, and a robust distillation loss function L is designed. The gradient value is calculated based on the robust distillation loss function L, and then the gradient value is back-propagated to the seed neural network model to train the seed neural network model. The obtained seed network model will no longer be controlled by the backdoor attack and can perform image recognition tasks normally.

[0012] According to the correlation between multiple neural network image recognition models in the distributed cluster, the neural network model with dense ranking connections is selected as the seed neural network model, which significantly improves the efficiency of erasing backdoor attacks in the distributed cluster and avoids the cross-implantation of backdoor triggers in the neural network model in the distributed cluster. By summing the attention activation graph of the seed neural network model, we can get To erase the backdoor trigger of another seed neural network model. In addition, by measuring the Euclidean distance and cosine distance of the attention activation maps of the two seed neural network models, a robust distillation loss function L is proposed, which can achieve the goal of erasing backdoor attacks while inheriting the image recognition ability of the original neural network, and significantly improve the defense efficiency of the neural network model cluster against backdoor attacks.

[0013] In step S1, the specific implementation process of selecting the ranked densely connected neural network model as the seed neural network model includes:

[0014] Based on the PageRank algorithm, the plurality of neural network models are sorted according to the numerical values ​​of the connections, and the top 10% are selected as seed neural network models, and added to the seed neural network model set;

[0015] The seed neural network models in the seed neural network model set are divided into several groups, and at most one neural network model is selected from each group as the seed neural network model.

[0016] The PageRank algorithm is used to erase backdoor attacks on multiple neural network image recognition models in distributed clusters. Compared with the random erasure method, the erasure efficiency of distributed clusters against backdoor attacks is significantly improved. By selecting seed neural network models, the cross-implantation of backdoor triggers into neural network models in distributed clusters is avoided.

[0017] The seed neural network model v is described by the following expression:

[0018] S[k][b]=max{S[k-1][b], S[k-1][b-Time(v)]+σ(v)|v∈group(k)};

[0019] Among them, b is a hyperparameter; S[k][b] represents the erasure range of the seed neural network model set found in the first k groups when the erasure time is b; Time(v) represents the time required for the seed neural network model v to be erased by another neural network model to trigger the backdoor, σ(v) is the final erasure range of the seed neural network model v; group(k) represents the seed neural network model set.

[0020] The erasing range of each seed neural network model set is dynamically limited by S[k][b]. Compared with specifying a specific erasing range, the present invention can dynamically plan the most efficient erasing efficiency.

[0021] In step S2, the attention activation map A l The expression is:

[0022]

[0023] in represents the ith channel of the lth layer of the attention activation map, |·| represents the absolute value function; C represents the number of channels of the attention activation map; represents the summation function, is the channel summation function.

[0024] The attention activation map shows which area of ​​the image the neural network image recognition model pays more attention to after learning, which is the embodiment of the learning results of the seed neural network model. to erase the backdoor trigger of another seed neural network model.

[0025] In step S3, the expression of the robust distillation loss function L is:

[0026]

[0027] Where (x, y)~D represents the image x and its corresponding label y extracted from the subset D of image data that has not been implanted with a backdoor trigger; E[·] represents the expected function; L kd_e (·) is expressed as the Euclidean distance between the activation attention maps of the two seed neural network models: L kd_c (·) is expressed as the cosine distance of the activation attention maps of the two seed neural network models: ||·|| 2 is the L2 norm, Respectively represent the activation attention maps of the two seed neural network models at layer l; is the channel summation function. It measures the classification error of one of the two seed neural network models, A. α and β are hyperparameters that control the intensity of attention distillation. J represents the number of neural network layers for obtaining the activation attention map.

[0028] The present invention proposes a robust distillation loss function, which can achieve the goal of erasing backdoor attacks while inheriting the image recognition capability of the original neural network by measuring the Euclidean distance and cosine distance of the attention activation maps of two seed neural network models.

[0029] As an inventive concept, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the steps of the above method of the present invention.

[0030] As an inventive concept, the present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon; the computer program / instruction, when executed by a processor, implements the steps of the method of the present invention.

[0031] As an inventive concept, the present invention also provides a computer program product, including a computer program / instruction; when the computer program / instruction is executed by a processor, the steps of the method of the present invention are performed.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention proposes a swarm intelligence algorithm for the first time for backdoor attacks based on multiple neural network image recognition models in a distributed cluster, and significantly improves the defense efficiency of the neural network image recognition model cluster against backdoor attacks. The swarm intelligence image recognition backdoor defense algorithm proposed by the present invention can be based on the association relationship between multiple neural network models in a distributed cluster, and by selecting a neural network model with dense ranking connections as a seed neural network model, the efficiency of erasing backdoor triggers between models in a distributed cluster reaches the current optimal level. In addition, for every two seed neural network models, the attention distillation representation is used to perform a distillation operation on the two seed neural network models, and a robust distillation loss function L is designed. Compared with the existing method, the obtained seed network model will be free from the control of the backdoor attack and can perform image recognition tasks normally. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A connection diagram between the neural network image recognition models of the distributed cluster according to an embodiment of the present invention;

[0034] Figure 2 The efficiency comparison between swarm intelligence and random erasure in the embodiment of the present invention is shown;

[0035] Figure 3 This is a backdoor erasure network architecture based on the neural distillation strategy in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention sets the defense scenario as a distributed cluster environment with multiple models. In which, due to the incremental training of multiple models, multiple image recognition models in a cluster are infected by poisoned data produced by malicious attackers. The goal of the present invention is to erase the backdoor triggers of the models in the cluster when different types of backdoor triggers are implanted in the distributed cluster of multiple neural network models, while maintaining the performance of image data that has not been implanted with backdoor triggers. The connection diagram between the neural network image recognition models in the distributed cluster of the present invention, such as Figure 1 shown.

[0037] The present invention proposes a swarm intelligence algorithm and a neural distillation strategy to improve the erasure accuracy and efficiency of multiple neural network models in a distributed cluster against backdoor attacks.

[0038] Considering that the present invention is aimed at erasing backdoor attacks on multiple neural network models in a distributed cluster, in order to improve the erasing efficiency of distributed clusters against backdoor attacks. Compared with the backdoor attack of randomly erasing neural network models, the present invention proposes a swarm intelligence algorithm to select seed neural network models to significantly improve the erasing efficiency of backdoor triggers. The efficiency comparison between the swarm intelligence of the present invention and random erasure is shown in Figure 2. Figure 2 shown.

[0039] The steps of the swarm intelligence algorithm are as follows:

[0040] First, based on PageRank [4] The algorithm sorts multiple neural network models in the distributed cluster according to the numerical value of the connection, and selects the top 10% as seed neural network models. Then, the seed neural network model set is divided into several groups. Since the distance between neural network models in the same group does not exceed 2, the mutual influence between models in the group is very strong. In addition, this only considers the influence value of this model on its close neighbors, reducing the computational workload.

[0041] Then, at most one model is selected from each group as the seed neural network model. Since the models in the group have a strong influence on each other and the seed neural network model can be propagated multiple times, the probability of other neural network models in the group being erased will be very high. However, the neural network model selected as the seed must meet two conditions: 1) The sum of the erasure time of all seed neural network models cannot exceed the time budget B of the erasure cluster backdoor attack; 2) The reachable erasure range of the seed neural network model set is the widest. The strategy for selecting the seed neural network model can be described by the following expression:

[0042] S[k][b]=max{S[k-1][b],S[k-1][b-Time(v)]+σ(v)|v∈group(k)} (1)

[0043] Among them, b is a hyperparameter; S[k][b] represents the erasure range of the seed neural network model set found in the first k groups when the erasure time is b; Time(v) represents the time required for the seed neural network model v to be erased by another neural network model to trigger the backdoor, σ(v) is the final erasure range of the seed neural network model v; group(k) represents the seed neural network model set.

[0044] In simple terms, when the backdoor erasure time budget is b, whether to search for a seed neural network model in the kth group. If, under the control of the backdoor erasure time b, the influence range of the seed neural network model set found in the first k-1 groups is greater than the erasure range of the seed neural network model set found in the first k groups, then it will be searched in the first k-1 groups, and the function becomes S[k][b]. Otherwise, a seed neural network model v will be searched in the kth group, and then the remaining neural network models will be searched in the first k-1 groups. At this time, the backdoor erasure time budget value needs to subtract v's cost Time(v), and the erasure range of the entire seed neural network model set needs to be added with the propagation range σ(v) of node v.

[0045] After the seed neural network model is selected through the swarm intelligence algorithm, the neural distillation strategy is executed on each two seed neural networks to achieve the erasure of the backdoor trigger. The backdoor erasure network architecture based on the neural distillation strategy of the present invention is as follows: Figure 3 shown.

[0046] For simplicity of description, the two seed neural network models are called model A and model B. First, for each layer of the neural network model, an attention activation map A that is robust to backdoor attacks is extracted. l Specifically, given a model A, its layer l activation output A l ∈R C×H×W , where R represents the dimension, C, H and W represent the channel, height and width of the attention activation map respectively. The present invention uses the absolute value of each element in the mapping to represent the importance of the element to the final output. Therefore, this mapping function can be constructed by calculating the statistics of the values ​​across these channel dimensions. The present invention applies the following mapping function:

[0047]

[0048] in represents the ith channel of the lth layer of the attention activation map, and |·| represents the absolute value function. The present invention adopts the channel summation function To activate all channels in each layer and generate an activation attention map representing the current layer. The activation attention map represented by the function has rich feature information. In order to enable the neural network image recognition to inherit the feature information to the greatest extent during distillation, the present invention proposes a robust distillation loss function based on the attention activation map to achieve the goal of erasing backdoor attacks while inheriting the recognition ability of the original neural network.

[0049] Therefore, after obtaining the activation attention map A of each layer l ∈R C×H×WAfter that, the activation attention graphs of model A and model B are calculated and input into the distillation loss to calculate the loss value. The details are as follows:

[0050]

[0051] The distillation loss can be expressed as the Euclidean distance L between the activation attention maps of model A and model B. kd_e (·) and cosine distance L kd_c (·),||·|| 2 is the L2 norm, Represent the activation attention maps of model A and model B at layer l respectively. It represents the channel summation function. It is worth mentioning that the attention activation map shows which area of ​​the image the neural network image recognition model pays more attention to after learning. It is the embodiment of the learning results of the seed neural network model. The present invention obtains to erase the backdoor trigger of another seed neural network model.

[0052]

[0053] Therefore, the overall training loss of the present invention is the sum of the cross entropy (CE) loss and the distillation loss. Where (x, y) ~ D represents the image x and its corresponding label y extracted from the subset D of image data that has not been implanted with the backdoor trigger; E[·] represents the expected function; L kd_e (·) represents the Euclidean distance between the activation attention maps of the two seed neural network models; L kd_c (·) is represented as the cosine distance of the activation attention maps of the two seed neural network models; ‖·‖ 2 is the L2 norm, Respectively represent the activation attention maps of the two seed neural network models at layer l; is the channel summation function. It measures the classification error of one of the two seed neural network models, A. α and β are hyperparameters that control the strength of attention distillation. J represents the number of neural network layers that obtain the activation attention map. The above loss function refers to model A distilling model B so that the backdoor trigger in model B is erased. When using model B to distill model A, you only need to swap the positions of model A and model B.

[0054] This paper proposes a swarm intelligence algorithm to optimize it to improve the efficiency of distributed clusters in erasing backdoor attacks. Then, attention distillation is used to represent [3] Extract the attention activation map that is robust to image recognition backdoor attacks, and finally, achieve the goal of erasing backdoor attacks by designing a robust distillation loss function. The specific process is as follows:

[0055] Step 1: Based on the association between multiple neural network image recognition models in a distributed cluster, use the PageRank proposed by Google [4] The algorithm selects the neural network model with dense ranking connections as the seed neural network model. For example, a server is connected to 5 servers in the entire distributed cluster, which is counted as 5. All servers in the distributed cluster are sorted according to the numerical value of the connection, and the top 10% are selected as the seed neural network model to improve the efficiency of erasing backdoor attacks by multiple neural network models in the distributed cluster.

[0056] Step 2: For each of the two seed neural network models, use the attention distillation representation to perform a distillation operation on the two seed neural network models to extract the attention activation map A that is invalid for the backdoor attack on the seed neural network model l , and use the channel sum function The attention activation maps of multiple channels are summed up to obtain To achieve erasing the backdoor triggers of the two seed neural network models;

[0057] Step 3: Use the activation attention map to sum multiple channels The Euclidean distance and cosine distance of the activation attention maps of the two seed neural network models are measured, and a robust distillation loss function is designed, the gradient value is calculated based on the robust distillation loss function, and then the gradient value is back-propagated to the seed neural network model [5] , train the seed neural network model, and the obtained seed network model will no longer be controlled by the backdoor attack and can perform the image recognition task normally.

[0058] The present invention evaluates the performance of the image recognition backdoor defense method based on swarm intelligence. The performance against six backdoor attacks is evaluated using two indicators, attack success rate (ASR) and classification accuracy (ACC), and compared with the other two existing backdoor defense methods, as shown in Table 1.

[0059] Table 1 Comparison of the defense performance of the proposed method and the latest backdoor defense method on the CIFAR-10 dataset

[0060]

[0061] Experiments show that the image recognition backdoor defense method based on swarm intelligence proposed in this paper significantly reduces the average attack success rate (ASR) from nearly 100% to 12.83%.

[12] and MCR

[13] The average attack success rate (ASR) can only be reduced to 37.36% and 25.59% respectively. Under the six attacks, it is fully proved that the backdoor defense method based on swarm intelligence image recognition proposed by the present invention has practical value in practice.

[0062] References:

[0063] [1] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.

[0064] [2]Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and XingjunMa. Neural attention distillation: Erasing backdoor triggers from deep neural networks. arXiv preprint arXiv:2101.05930, 2021.

[0065] [3] Yuenan Hou, Zheng Ma, Chunxiao Liu, and Chen Change Loy. Learning lightweight lane detection cnns by self attention distillation. In Proceedings of the IEEE / CVF International Conference on Computer Vision, pages 1013-1021, 2019.

[0066] [4]Wenpu Xing and Ali Ghorbani.Weighted pagerank algorithm.InProceedings.Second Annual Conference on Communication Networks and ServicesResearch,2004.,pages 305-314.IEEE,2004.

[0067] [5]Rumelhart D E,Hinton G E,Williams R J.Learning representations byback-propagating errors.nature,1986,323(6088):533-536.

[0068] [6]Tianyu Gu,Brendan Dolan-Gavitt,and Siddharth Garg.Badnets:Identifying vulnerabilities in the machine learning model supply chain.arXivpreprint arXiv:1708.06733,2017.

[0069] [7]Yingqi Liu.Trojaning attack on neural networks.In NDSS,2018

[0070] [8]Xinyun Chen,Chang Liu,Bo Li,Kimberly Lu,and Dawn Song.Targetedbackdoor attacks on deep learning systems using data poisoning.arXiv preprintarXiv:1712.05526,2017.

[0071] [9]Alexander Turner,Dimitris Tsipras,and Aleksander Madry.Clean-labelbackdoor attacks.https: / / people.csail.mit.edu / madry / lab / ,2018.

[0072]

[10] Mauro Barni,Kassem Kallas,and Benedetta Tondi.A new backdoorattack in cnns by training set corruption without label poisoning.In 2019IEEE International Conference on Image Processing(ICIP),pages 101-105.IEEE,2019.

[0073]

[11] Yunfei Liu,Xingjun Ma,James Bailey,and Feng Lu.Reflectionbackdoor:A natural backdoor attack on deep neural networks.In EuropeanConference on Computer Vision,pages 182-199.Springer,2020.

[0074]

[12] Kang Liu,Brendan Dolan-Gavitt,and Siddharth Garg.Fine-pruning:Defending against backdooring attacks on deep neural networks.InInternational Symposium on Research in Attacks,Intrusions,and Defenses,pages273-294.Springer,2018.

[0075]

[13] Pu Zhao,Pin-Yu Chen,Payel Das,Karthikeyan Natesan Ramamurthy,andXue Lin.Bridging mode connectivity in loss landscapes and adversarialrobustness.In International Conference on Learning Representations,2019.

Claims

1. An image recognition backdoor defense method based on swarm intelligence, characterized in that: The following steps are involved: S1. Based on the correlation between multiple neural network image recognition models in the distributed cluster, the densely connected neural network model is selected as the seed neural network model; for each two seed neural network models, the attention distillation representation is used to perform a distillation operation on the two seed neural network models to extract the attention activation map A that is invalid for the backdoor attack on the seed neural network model l ; S2, by using the channel summation function Attention activation map for multiple channels A l Perform the sum operation and get the sum result S3. Utilization Measuring the Euclidean distance and cosine distance of the activation attention maps of the two seed neural network models, and designing a robust distillation loss function L, calculating a gradient value based on the robust distillation loss function L, and using the gradient value to backpropagate the seed neural network model to train the seed neural network model; The seed neural network model v is described by the following expression: S[k][b]=max{S[k-1][b], s[k-1][b-Time(v)]+σ(v)|v∈group(k)}; Where b is a hyperparameter; S[k]][b] represents the erasure range of the seed neural network model set found in the first k groups when the erasure time is b; Time(v) represents the time it takes for the seed neural network model v to be erased by another neural network model to trigger the backdoor, and σ(v) is the final erasure range of the seed neural network model v; group(k) represents the seed neural network model set; In step S3, the expression of the robust distillation loss function L is: Where (x, y)~D represents the image x and its corresponding label y extracted from the subset D of image data that has not been implanted with a backdoor trigger; E[·] represents the expected function; L kd_e (·) is expressed as the Euclidean distance between the activation attention maps of the two seed neural network models: L kd_c (·) is expressed as the cosine distance of the activation attention maps of the two seed neural network models: ||·||2 is the L2 norm, Respectively represent the activation attention maps of the two seed neural network models at layer l; is the channel summation function; It measures the classification error of one of the two seed neural network models, A. α and β are hyperparameters that control the intensity of attention distillation. J represents the number of neural network layers for obtaining the activation attention map.

2. The image recognition backdoor defense method based on swarm intelligence according to claim 1 is characterized in that: In step S1, the specific implementation process of selecting the densely connected neural network model as the seed neural network model includes: Ranking the multiple neural network models based on the PageRank algorithm, selecting densely connected neural network models in proportion, and adding them to the seed neural network model set; The seed neural network models in the seed neural network model set are divided into several groups, and at most one neural network model is selected from each group as the seed neural network model.

3. The image recognition backdoor defense method based on swarm intelligence according to claim 1 is characterized in that: In step S2, the attention activation map A l The expression is: in, represents the ith channel of the lth layer of the attention activation map, |·| represents the absolute value function; C represents the number of channels of the attention activation map; represents the summation function, is the channel summation function.

4. The image recognition backdoor defense method based on swarm intelligence according to any one of claims 1 to 3, characterized in that: The image is used as input to the trained seed neural network model to identify the image category.

5. A computer device comprising a memory, a processor and a computer program stored in the memory; characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program / instruction stored thereon; characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer program product comprising a computer program / instructions; characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images