Multi-mode invisible backdoor attack method and system based on countermeasure disturbance and medium

By generating image structure triggers with invisible perturbations and designing a loss function for self-supervised learning, the problem of balancing concealment and success rate in self-supervised learning backdoor attacks is solved, and efficient backdoor attacks are achieved in various models.

CN118940818APending Publication Date: 2024-11-12HUNAN UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410961017.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In existing self-supervised learning backdoor attack methods, it is difficult to balance the success rate and stealth of backdoor attacks. Fixed visible pixel triggers reduce stealth and have a high false trigger rate, while steganographic string triggers are not practical in a wide range of scenarios.

Method used

By generating image structure triggers with invisible perturbations, extracting and coloring image edges using RCF, and designing a loss function by combining generative adversarial networks and SimCLR contrastive learning, the training process is controlled to implant a backdoor, generating triggers specific to the original image, thereby improving stealth and maintaining the success rate of attacks.

Benefits of technology

While reducing the false trigger rate, it maintains a high backdoor attack success rate and is applicable to ResNet-18, ResNet-50 and CLIP multimodal models, achieving good performance and practicality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure BDA0004950271550000031
    Figure BDA0004950271550000031
  • Figure BDA0004950271550000033
    Figure BDA0004950271550000033
  • Figure BDA0004950271550000034
    Figure BDA0004950271550000034
Patent Text Reader

Abstract

The invention discloses a self-supervised learning backdoor attack method and system based on an image structure trigger and a medium, and the method comprises the steps: firstly, extracting the edge of a training image through an image edge extraction method, carrying out the coloring, converting the edge into invisible disturbance through a generative adversarial model, and hiding the invisible disturbance in an image edge structure, and obtaining a poisoning image; then, designing a loss function of an implantation backdoor by utilizing comparative learning in self-supervised learning, and realizing backdoor implantation by minimizing cosine similarity between poisoning data and target type clean data; and finally, activating the backdoor in the target model by using a poisoning sample containing an image structure trigger to complete backdoor attack. Invisible disturbance is used as a trigger, poisoning data and corresponding clean data are difficult to distinguish, and the concealment of the rear door can be effectively improved. Meanwhile, the image structure can effectively protect semantic features of the trigger, and the success rate of backdoor attack is maintained at a high level.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer science and technology, and in particular to a self-supervised learning backdoor attack method, system and medium based on image structure trigger. Background Art

[0002] In recent years, the amount of data on the Internet has grown explosively. In order to efficiently utilize the massive amount of data on the Internet and reduce the cost of manual labeling, self-supervised learning has gradually become the mainstream training method. Self-supervised learning has the ability to extract and learn features from unlabeled data, but it requires a lot of computing power as support. Therefore, users usually use cloud services to train self-supervised models, or rely on fine-tuning pre-trained models to complete the training of downstream tasks. However, this training method that relies on cloud services has privacy leakage problems. Users have insufficient control over the model training process, which can easily lead to the model being threatened by backdoor attacks.

[0003] Backdoor attack technology has made significant progress in the field of supervised learning, and people have begun to turn their attention to the more complex and challenging field of self-supervised learning. There are few existing studies on backdoor attacks in self-supervised learning, which are mainly focused on computer vision models. However, with the promotion of large language models and multimodal large models such as ChatGPT, Sora, and Grok-1, more and more self-supervised learning models are used in intelligent manufacturing, autonomous driving, medical testing, and industrial production. Ensuring the security of self-supervised learning has gradually become an important research topic to protect social production and people's lives, and research on backdoor attacks on self-supervised learning is imminent.

[0004] There are few studies on backdoor attacks in self-supervised learning, and existing research work mainly focuses on the field of computer vision models. The attacker embeds visible pixel triggers or steganographic string triggers in the training set, and uses the characteristics of self-supervised learning to construct a loss function for backdoor implantation. Because the dataset used in self-supervised learning lacks manual annotations, the attacker implants the backdoor by indirectly constructing a way to associate the trigger with the target class. However, this attack method has obvious flaws.

[0005] On the one hand, although attackers can achieve a high attack success rate by using embedded triggers of visible pixels, the fixed visible trigger mode not only greatly reduces the concealment of the backdoor, but also leads to a high false trigger rate (FAR, False Activation Rate), that is, the clean model has a high proportion of incorrect recognition and misclassification of poisoned data.

[0006] On the other hand, although the method of steganographically writing fixed characters into images improves the concealment of the backdoor and reduces the false trigger rate to a certain extent, the attack performance is greatly reduced. In addition, because the steganographic string is difficult to maintain the semantics of the trigger, it is not practical enough in a wider range of scenarios. Summary of the invention

[0007] Technical solution of the present invention The present invention aims at the problem that it is difficult to balance the success rate and concealment of backdoor attacks in self-supervised learning backdoor attack methods. When generating image triggers, the backdoor triggers are specific to the original clean images, that is, the backdoor triggers contained in any two modified training images are different. The triggers are converted into invisible perturbations by the generation model of poisoned data and hidden in the image structure. The triggers in the form of invisible perturbations can effectively improve the concealment of the backdoor, and the image structure can effectively protect the semantic features of the trigger activating the backdoor. Through training, the cosine similarity between the poisoned image and the clean image of the target class is maximized, and the backdoor is implanted into the model. When the verification image contains a trigger with the same design criteria, the backdoor in the model will be activated, and the model will misclassify the poisoned data into the target class.

[0008] The present invention provides the following technical solutions:

[0009] On the one hand, a self-supervised learning backdoor attack method based on image structure triggers includes:

[0010] Construct poisoned images: Use the RCF scheme to extract the edges of clean images and color them as the input of the poisoned data generation adversarial model. Select UNet as the generator and PatchGAN as the discriminator, and add an auxiliary guidance network after the generation model to guide the generation model to learn trigger features. Finally, only the generator is retained as the poisoned data generation model;

[0011] Constructing a loss function for implanting a backdoor: Use SimCLR contrastive learning in self-supervised learning to design a loss function for implanting a backdoor, including one validity loss and two concealment losses. In the process of minimizing the validity loss, the feature vectors generated by the backdoor model for the poisoned data and the clean data of the target class become more and more similar. In the process of minimizing the concealment loss, the feature vectors generated by the clean model and the backdoor model for the clean data become more and more similar;

[0012] Training the target model: Use the constructed poisoned images and loss function to control the training process. During the training process, the cosine similarity between the poisoned images and the clean images of the target class is continuously improved, indirectly building a malicious association between the trigger and the target class, while maintaining the accuracy of the backdoor model in recognizing clean images. By controlling the training process, the attacker can implant a backdoor into the pre-trained model or implant a backdoor during downstream tasks. This method is used to implant a backdoor into the target model ResNet-18, ResNet-50, and the multimodal CLIP model.

[0013] The pre-trained model uses CIFAR10 and STL10 to train ResNet-18 from scratch, and obtains ResNet-50 and CLIP pre-trained models pre-trained using the ImageNet dataset from open source websites.

[0014] An attacker controlling the training process means that the attacker pretends to be a third-party cloud service provider and controls the entire training process by modifying the data submitted by the user or modifying the training loss function.

[0015] Combined with the characteristics of deep learning models learning image features, when generating triggers, the backdoor triggers are specific to the original clean images, that is, the backdoor triggers contained in any two modified training images are different. In traditional backdoor attacks against self-supervised learning, the triggers in the poisoned images often have the same form, and different poisoned images contain the same or similar triggers. For example, insert a white square in the lower right corner of the image or write the same string into the image. Therefore, the present invention innovatively proposes to use the image edge structure to hide the trigger information, such as using the RCF image edge extraction method, coloring after extracting the image edge, and obtaining the original image-specific trigger. Using the generative adversarial network, the trigger is converted into an invisible perturbation and hidden in the edge structure of the image. Through the loss function designed by SimCLR comparative learning, the training process of the three target models ResNet-18, ResNet-50 and CLIP can be controlled, so that the poisoned image containing the trigger can easily mislead the model and lead to the generation of wrong decisions.

[0016] Furthermore, the original trigger is generated by:

[0017]

[0018] Among them, x i is a clean image, RCF(x i ) represents the extraction of the edge of the clean image. Indicates the coloring operator, C (r,g,b) Indicates that the rgb color can be any value, t i Represents the original trigger.

[0019] Furthermore, the loss function of the poisoned data generator is designed by the L1 norm, and its formula is:

[0020]

[0021] Among them, L G is the loss function of the poisoned data generator, X represents the clean training dataset, x i * Represents clean data x i Corresponding poisoning data.

[0022] Use the L1 norm as f G The loss function of () can ensure that the generator generates clearer poisoned images, improve the robustness and training efficiency of the model, and effectively strike a balance between image quality and the invisibility of stego information.

[0023] Furthermore, the L1 norm is mixed into the cross entropy loss to improve the discriminator f D () to further reduce the discrimination performance of f G () generated poisoned image With the clean image x i The difference between them greatly improves the concealment of the backdoor in the poisoned image. The objective function of the discriminator is:

[0024]

[0025] Among them, L D is the objective function of the discriminator, f G (x i ,t i ) is the process by which the generator generates poisoned images using clean images and original triggers, X * It is a poisoned dataset.

[0026] Furthermore, an auxiliary guidance network structure is added after the generation network to guide the poisoned image generation model to learn the features of the trigger, ensuring that the generated poisoned data can successfully activate the backdoor. Its loss function is:

[0027]

[0028] Among them, L lead is the loss function of the auxiliary guidance network, f lead () is the process of assisting the network to process data, X c is the clean image x i 's mapping.

[0029] Here, we use the L2-norm to design the loss function of the auxiliary guidance network. Because the derivative of the L2-norm is continuous everywhere, using the L2-norm can make the guidance process more stable, ensuring that the model can fuse the trigger with the clean image and extract the trigger features from the poisoned image, while ensuring that the model cannot extract features from the clean image.

[0030] Furthermore, the process of constructing the loss function of the backdoor implant is as follows:

[0031] The poisoned image generation model is used to generate poisoned images and implant a backdoor into the target model. The SimCLR contrastive learning algorithm is used to design a loss function for implanting the backdoor, including an effectiveness loss function and two concealment loss functions. The effectiveness loss function and the concealment loss function are:

[0032] (1) The validity loss function is defined as:

[0033]

[0034] Among them, L eff is the effectiveness loss function, |D| is the amount of training data selected by the attacker for implanting the backdoor, sim() is the function for calculating the cosine similarity of two feature vectors, and F * () is the process of the backdoor model converting image data into feature vectors, x i j is the jth image data of the i-th target class, x * is the poisoned image, n i is the amount of test data, and each test class contains m test data.

[0035] L eff The loss-constrained backdoor model is the target class data x in D i j and poisoned data x * The act of generating vectors encourages the backdoor model to generate more similar feature vectors for the two. eff In the process, the backdoor model is x i j and x * The cosine similarity of the generated feature vectors gradually increases.

[0036] (2) The first hidden loss function is defined as:

[0037]

[0038] Among them, L con1 is the first hidden loss function, F() is the process of converting image data into feature vector by the clean target model, and D iClean image data for the target class.

[0039] L con1 Constrain the backdoor model and the clean target model to generate vectors for the target class dataset, so that both are D i The data in the minimization L con1 In the process, the backdoor model and the clean target model are used to compare the target class dataset D i The recognition accuracy is getting closer.

[0040] (3) The second hidden loss function is defined as:

[0041]

[0042] Among them, L con2 is the second concealment loss function, and x is the image data in the clean dataset D selected by the attacker for implanting the backdoor.

[0043] L con2 Constrain the backdoor model and the clean target model to generate feature vectors for the image data in D, and improve the recognition accuracy of the backdoor model and the clean target model for various types of data in D. con2 In the process, the backdoor model and the clean target model gradually approach each other in terms of the accuracy of distinguishing the data in D.

[0044] Furthermore, the final loss function is the effectiveness loss function L eff And the hidden loss function L con The linear combination of , the final loss function is:

[0045] L=L eff +α·L con1 +β·L con2

[0046] Among them, α and β are hyperparameters, α controls the downstream task test dataset D i Impact on model accuracy, β controls the impact of the dataset D used to implant the backdoor on the model accuracy. By default, α=β=1.

[0047] In a second aspect, a system using the above-mentioned self-supervised learning backdoor attack method based on image structure, comprising:

[0048] Poisoned image construction unit: The edges of the clean image are extracted and colored through the RCF scheme as the input of the poisoned data generation adversarial model. UNet is selected as the generator, PatchGAN as the discriminator, and an auxiliary guidance network is added after the generation model to guide the generation model to learn trigger features.

[0049] Backdoor implantation construction unit: The loss function for implanting the backdoor is designed through SimCLR contrastive learning in self-supervised learning, including one validity loss and two concealment losses. In the process of minimizing the validity loss, the feature vectors generated by the backdoor model for the poisoned data and the clean data of the target class become more and more similar. In the process of minimizing the concealment loss, the feature vectors generated by the clean model and the backdoor model for the clean data become more and more similar;

[0050] Training target model unit: Control the training process through poisoned images and loss functions, continuously improve the cosine similarity between poisoned images and clean images of the target class, indirectly build malicious associations between triggers and target classes, and maintain the accuracy of clean image recognition by the backdoor model. By controlling the training process, the attacker can implant backdoors into the pre-trained model or implant backdoors during downstream tasks. This method can be used to implant backdoors into the target model ResNet-18, ResNet-50, and multimodal CLIP models.

[0051] In a third aspect, an electronic device at least comprises:

[0052] one or more processors;

[0053] a memory storing one or more computer programs;

[0054] The processor calls a computer program to execute:

[0055] The above steps are a multimodal invisible backdoor attack method based on adversarial perturbation.

[0056] In a fourth aspect, a computer-readable storage medium stores a computer program, wherein the computer program is called by a processor to execute:

[0057] The above is the steps of a self-supervised learning backdoor attack method based on image structure.

[0058] Beneficial Effects

[0059] The technical solution of the present invention provides a self-supervised learning backdoor attack method, system, device and medium based on image structure trigger. Aiming at the problem that it is difficult to balance the concealment and effectiveness of the backdoor in the self-supervised learning backdoor attack method, a self-supervised learning backdoor attack method SSL-BIST based on image structure trigger is proposed. Firstly, the image edge is extracted and colored by the image edge extraction method to obtain the original trigger specific to the image, and the triggers between different images are different; then, the original trigger is converted into an invisible perturbation form by generating an adversarial network and hidden in the edge structure of the image to obtain poisoned data; secondly, the effectiveness loss and concealment loss functions of the implanted backdoor are designed by the SimCLR contrast learning algorithm in self-supervised learning, and the backdoor is implanted in the process of controlling the model training; the trigger with the same design criteria is added to the input of the target model implanted with the backdoor to realize the backdoor attack on the self-supervised learning model. The experimental results show that the attack method proposed by the present invention is effective, and the success rate of the backdoor attack is maintained at a high level while greatly reducing the false trigger rate.

[0060] The main advantages of the present invention are as follows:

[0061] (1) When generating poisoned images, the extracted image edges are colored and converted into invisible perturbations, which are hidden in the edge structure of the image. The trigger in the form of invisible perturbations can effectively improve the concealment of the backdoor, and the image structure can effectively maintain the semantic characteristics of the trigger activating the backdoor.

[0062] (2) In the process of backdoor implantation, SimCLR in self-supervised learning is used to design the effectiveness loss and concealment loss functions of the implanted backdoor. By maximizing the cosine similarity between the poisoned image and the clean image of the target class, the malicious association between the poisoned image and the target class is indirectly constructed to achieve backdoor implantation.

[0063] (3) The backdoor attack invention can significantly reduce the false trigger rate while maintaining a high success rate of the backdoor attack. At the same time, the invention can be applied not only to simple models such as ResNet-18, but also to more complex ResNet-50 and CLIP multimodal models to achieve good performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 Schematic diagram of the original trigger generation and poisoned data generation process;

[0065] Figure 2 This is a schematic diagram of the backdoor implantation process;

[0066] Figure 3 A schematic diagram comparing the clean image, the patch trigger poisoned data, the string steganography trigger poisoned data and the poisoned data of the present invention;

[0067] Figure 4 Different indicators for evaluating the quality of poisoned data in different schemes, where (a) is PSNR, (b) is SSIM, and (c) is LPIPS;

[0068] Figure 5 Different indicators for downstream tasks of ResNet-18 pre-trained on CIFAR10, where (a) is ASR, (b) is FAR, (c) is ACC, and (d) is B-ACC;

[0069] Figure 6 Different indicators of ResNet-18 pre-trained for STL10 when performing downstream tasks, where (a) is ASR, (b) is FAR, (c) is ACC, and (d) is B-ACC;

[0070] Figure 7 Indicators of different batch-sizes for downstream tasks of ResNet-18 pre-trained on CIFAR10, where (a) is ASR and (b) is FAR;

[0071] Figure 8 Indicators of different batch-sizes when ResNet-18 pre-trained on STL10 performs downstream tasks, where (a) is ASR and (b) is FAR;

[0072] Fig. 9 Different indicators for the pre-trained model ResNet-50 when performing downstream tasks, where (a) is ASR, (b) is FAR, (c) is ACC, and (d) is B-ACC;

[0073] Fig.10 Different indicators for the pre-trained model CLIP when performing downstream tasks, where (a) is ASR, (b) is FAR, (c) is ACC, and (d) is B-ACC;

[0074] Fig.11 The influence of the hyperparameters α and β on different indicators, where (a) is the influence of α on ASR and B-ACC, and (b) is the influence of β on ASR and B-ACC;

[0075] Fig.12 Schematic diagram of the effect of data processing on poisoned images generated by different schemes.

[0076] Fig.13 ASR schematic diagram of poisoned data in the CLIP multimodal model after data processing. DETAILED DESCRIPTION

[0077] The present invention is further described in detail below with reference to the accompanying drawings and embodiments:

[0078] Example 1

[0079] This embodiment provides a self-supervised learning backdoor attack method based on image structure trigger (SSL-BIST), which uses the structural features of the image and the features of self-supervised learning to design a self-supervised learning backdoor attack method that is both concealed and has strong attack capabilities, including:

[0080] like Figure 1 As shown, construct the poisoned image: use the RCF scheme to extract the edges of the clean image and color it as the input of the poisoned data generation adversarial model. Select UNet as the generator, PatchGAN as the discriminator, and add an auxiliary guidance network after the generation model to guide the generation model to learn the trigger features. Finally, only the generator is retained as the poisoned data generation model;

[0081] The scheme uses the RCF method to extract the image edge structure and fills C into the extracted image edge structure. (r ,g,b) The color value gets the image-specific raw trigger, where (r,g,b) can take any value. The trigger extraction and colorization process is:

[0082]

[0083] Among them, x i is a clean image, RCF(x i ) represents the extraction of the edge of the clean image. Indicates the coloring operator, C (r,g,b) Indicates that the rgb color can be any value, t i Represents the original trigger.

[0084] The original trigger and the clean image are used as the input of the poisoned data generator, and the loss function of the poisoned data generator is designed by the L1 norm, and its formula is:

[0085]

[0086] Among them, L G is the loss function of the poisoned data generator, X represents the clean training dataset, x i * Represents clean data x i Corresponding poisoning data.

[0087] Use L1 norm as f GThe loss function of () can ensure that the generator generates clearer poisoned images, improve the robustness and training efficiency of the model, and effectively strike a balance between image quality and the invisibility of stego information.

[0088] The training of the discriminator and the training of the generator form an adversarial relationship. The images generated by the generator hope to deceive the discriminator, while the discriminator tries its best to find the difference between the generated images and the clean images. Through this adversarial training method, the quality of the generated images is continuously improved, and the difference between them and the clean images is getting smaller and smaller.

[0089] For discriminator training, mix the L1 norm into the cross entropy loss to improve the discriminator f D () to further reduce the discrimination performance of f G () generated poisoned image With the clean image x i The difference between them greatly improves the concealment of the backdoor in the poisoned image. The objective function of the discriminator is:

[0090]

[0091] Among them, L D is the objective function of the discriminator, f G (x i ,t i ) is the process by which the generator generates poisoned images using clean images and original triggers, X * It is a poisoned dataset.

[0092] In order to ensure that the trigger features in the poisoned image can be successfully recognized by the model, an auxiliary guidance network structure is added after the generation network to guide the poisoned image generation model to learn the features of the trigger. The auxiliary guidance network also forms an adversarial relationship with the generator, ensuring that the model can extract the features of the trigger from the poisoned data, while failing to extract the features of the trigger from the clean image.

[0093] To ensure that the generated poisoned data can successfully activate the backdoor, the loss function of the auxiliary guidance network is:

[0094]

[0095] Among them, L lead is the loss function of the auxiliary guidance network, f lead () is the process of assisting the network to process data, X c is the clean image x i 's mapping.

[0096] Here, we use the L2-norm to design the loss function of the auxiliary guidance network. Because the derivative of the L2-norm is continuous everywhere, using the L2-norm can make the guidance process more stable, ensuring that the model can fuse the trigger with the clean image and extract the trigger features from the poisoned image, while ensuring that the model cannot extract features from the clean image.

[0097] like Figure 2 As shown in the figure, construct the loss function for implanting the backdoor: use SimCLR contrastive learning in self-supervised learning to design the loss function for implanting the backdoor, including one validity loss and two concealment losses. In the process of minimizing the validity loss, the feature vectors generated by the backdoor model for the poisoned data and the clean data of the target class become more and more similar. In the process of minimizing the concealment loss, the feature vectors generated by the clean model and the backdoor model for the clean data become more and more similar;

[0098] The poisoned image generation model is used to generate poisoned images, and the backdoor is implanted into the target model using the poisoned images. The attacker uses the SimCLR contrastive learning algorithm to design a loss function for implanting the backdoor, including one validity loss function and two concealment loss functions. The validity loss function and concealment loss function are:

[0099] (1) The validity loss function is defined as:

[0100]

[0101] Among them, L eff is the effectiveness loss function, |D| is the amount of training data selected by the attacker for implanting the backdoor, sim() is the function for calculating the cosine similarity of two feature vectors, and F * () is the process of the backdoor model converting image data into feature vectors, x i j is the jth image data of the i-th target class, x * is the poisoned image, n i is the amount of test data, and each test class contains m test data.

[0102] L eff The loss-constrained backdoor model is the target class data x in D i j and poisoned data x * The act of generating vectors encourages the backdoor model to generate more similar feature vectors for the two. eff In the process, the backdoor model is x i j and x * The cosine similarity of the generated feature vectors gradually increases.

[0103] (2) The first hidden loss function is defined as:

[0104]

[0105] Among them, L con1 is the first hidden loss function, F() is the process of converting image data into feature vector by the clean target model, and D i Clean image data for the target class.

[0106] L con1 Constrain the backdoor model and the clean target model to generate vectors for the target class dataset, so that both are D i The data in the minimization L con1 In the process, the backdoor model and the clean target model are used to compare the target class dataset D i The recognition accuracy is getting closer.

[0107] (3) The second hidden loss function is defined as:

[0108]

[0109] Among them, L con2 is the second concealment loss function, and x is the image data in the clean dataset D selected by the attacker for implanting the backdoor.

[0110] L con2 Constrain the backdoor model and the clean target model to generate feature vectors for the image data in D, and improve the recognition accuracy of the backdoor model and the clean target model for various types of data in D. con2 In the process, the backdoor model and the clean target model gradually approach each other in terms of the accuracy of distinguishing the data in D.

[0111] Through the effectiveness loss function and the two concealment loss functions, the final loss function of backdoor implantation is designed to transform the backdoor implantation problem into an optimization problem. The final loss function is the effectiveness loss function L eff And the hidden loss function L con The linear combination of , the final loss function is:

[0112] L=L eff +α·L con1 +β·L con2

[0113] Among them, α and β are hyperparameters, α controls the downstream task test dataset D i Impact on model accuracy, β controls the impact of the data set D used to implant the backdoor on the model accuracy. In this example, α=β=1.

[0114] By iteratively adjusting the network weights and parameters of the target models ResNet-18, ResNet-50, and CLIP, the model with the backdoor implanted is obtained.

[0115] The trained backdoor model is in the process of activating the backdoor to implement the backdoor attack:

[0116] (1) Input clean data

[0117] For clean inputs without any triggers, the model is expected to work normally and output the correct answer based on its training and learning. Despite the backdoor implanted in the VQA model, it still maintains its original functionality and accuracy and responds normally to clean inputs without activating triggers.

[0118] (2) Input a poisoned image containing an image structure trigger

[0119] When the attacker provides the model with a poisoned image with an image structure trigger, if the model misclassifies it into the target class pre-set by the attacker, it indicates that the backdoor attack is successful. In this case, the output of the target model implanted with the backdoor is successfully manipulated by the attacker because the input contains the trigger that activates the backdoor, causing the target model to deviate from its normal decision logic.

[0120] In the process of backdoor attack, the attacker has obvious advantages in hiding the trigger information by using the image structure. First, when the generated trigger is deeply integrated with the image, the trigger information appears in the form of invisible disturbance, which can effectively improve the concealment of the backdoor. Second, the trigger is hidden in the edge structure of the image, and the trigger information is scattered throughout the image. This scheme allows the trigger to remain semantically unchanged after common data processing and transformation, which can effectively maintain the performance of the backdoor attack. At the same time, the shallow network of DNN can easily capture low-level information such as image structure, but the final decision of DNN depends on the details of the image, which can maintain the accuracy of the backdoor model in distinguishing clean data. Finally, the attacker can also meet the needs of multi-target or multi-form backdoor attacks by changing different image edge extraction methods or changing the color value C(r,g,b) of the image edge filling, that is, the backdoor attack implemented by the image structure trigger has good practicality and scalability.

[0121] In combination with the characteristics of contrastive learning in self-supervised learning, a loss function for implanting backdoors is designed. In the process of using contrastive learning to train the model, the training set used does not have manually annotated label information. Contrastive learning obtains a lot of positive and negative data by performing data enhancement operations on the training set. Positive data refers to training data and corresponding enhanced data, and negative data refers to training data and non-corresponding enhanced data. Contrastive learning achieves feature learning of unlabeled data by continuously reducing the distance between positive data and increasing the distance between negative data. In the technical solution of the present invention, the attacker uses the SimCLR contrastive learning algorithm with a simple framework to implement backdoor implantation. In the process of backdoor implantation, the attacker uses the characteristics of SimCLR to continuously reduce the distance between poisoned data and clean data of the target class, indirectly construct a malicious association between poisoned data and the target class, and implement backdoor implantation. By training the target model in this way, poisoned images containing the same design criteria triggers can easily mislead the model, resulting in the generation of wrong decisions.

[0122] Take ResNet-18 as an example. Its training set usually consists of images of equal size. The images may not contain labels and are placed in different folders. An attacker named Rose downloaded a ResNet model from the cloud and tried to implant a backdoor in the ResNet model and then upload it to the cloud for users to download. The process of implanting the backdoor is as follows:

[0123] (1) Training the poisoned image generator: Use UNet as the poisoned data generator, PatchGAN as the discriminator, add an auxiliary guidance network, and select COCO2014 as the training data for the generator. First, use the RCF scheme to extract the edges of some images and color them as the input of the generator. Iterate 200 times, set the learning rate to 0.0002, and set the training batch to 256. The purpose of the iteration is to make the poisoned data generated by the poisoned data generator higher in quality, and the trigger is hidden in the edge structure of the image in a more invisible form. At the same time, the features of the trigger can be recognized by the model. After 200 iterations, a poisoned data generation model with better performance can be obtained.

[0124] (2) Target model training: Set a target label "airplane", select an image that is not in the airplane folder in the open source CIFAR10 dataset, use the poisoned image generator to create poisoned data, and use the open source dataset CIFAR10 as the training set for implanting the backdoor. Take poisoned image A as an example. Then select ResNet-18, ResNet-50 and CLIP as the target model, use the loss function implanted with the backdoor, maximize the cosine similarity between poisoned image A and the clean image in the "airplane" folder, iterate 200 times, set the learning rate to 0.0001, and set the training batch to 256. After completing the training, the attacker Rose uploads it to the cloud for users to download and deploy.

[0125] (3) Testing the target model: When the user downloads the backdoor pre-trained model in (2) for downstream tasks, the downstream tasks will inherit the backdoor. When the user uses a clean image in the "truck" class to test the model, the model will correctly identify it as "truck". The user will think that the model can meet the needs and start to use it, but the backdoor in the model is always likely to be triggered. The attacker can launch an attack on the system containing the target model according to the poisoned data generated in (1) above.

[0126] Experimental results analysis

[0127] 1. Preparation

[0128] (1) Experimental environment The experiment was conducted on the Ubantu18.04 system and the Intel(R) Xeon(R) Silver 4214R CPU, and the RTX 3090 GPU accelerator was used to improve the experimental efficiency. The experiment was based on Python 3.8 and open source code. The training of the poisoned data generation model used the Pytorch framework, and the training of implanting backdoors into the target model also used the PyTorch framework.

[0129] (2) Experimental datasets The experiments use CIFAR10, STL10, SVHN, GTSRB and COCO2014 datasets.

[0130] 1) CIFAR10 contains 60,000 32*32 color images in 10 categories, with 6k images in each category. It is a dataset with evenly distributed images of various categories.

[0131] 2) STL10 contains 113,000 96*96 color images from 10 categories. Each category has 1.3k labeled images and 100,000 unlabeled images. It is also a dataset with evenly distributed images of various categories.

[0132] 3) SVHN contains 6,30,420 32*32 color images, similar to the MINST dataset, with a total of ten categories from '0' to '9', but SVHN contains more noise and interference, and the distribution of various types of data is uneven. It is often used for digital recognition in real scenarios.

[0133] 4) COCO2014 contains 164,000 images of 80 categories with inconsistent sizes. This dataset collects a large number of real images from the real world and is an open source dataset with diverse target categories and a wide range of application scenarios.

[0134] (3) Evaluation indicators

[0135] 1) ASR (Attack success rate): Attack success rate. The downstream model containing the backdoor will misclassify the poisoned image into the target category pre-specified by the attacker. ASR is the ratio of successfully misclassified poisoned test data to all poisoned data. This indicator should be as high as possible.

[0136] 2) FAR (False Activation Rate): False trigger rate. The trigger should reduce the impact on the accuracy of the clean model. The clean model will not misclassify or identify the poisoned data. This indicator should be as low as possible.

[0137] 3) ACC (Accuracy): The prediction accuracy of clean images. The classification accuracy of the clean downstream model for clean data is determined by many factors, including training data set, training batch, learning rate, number of iterations, etc.

[0138] 4) B-ACC (Backdoor-Accuracy): The prediction accuracy of clean images on the backdoor model. The downstream task model that inherits the backdoor should maintain a high accuracy for clean data, and this indicator should be as close to ACC as possible.

[0139] (4) Attack Settings

[0140] In the poisoned data generation model, the UNet framework is used as the generator and the PatchGAN framework is used as the discriminator to generate poisoned images. In the experiment, ResNet-18, ResNet-50 and CLIP multimodal models are used as target models to explore the attack effect of the SSL-BIST framework.

[0141] When generating poisoned data, the technical solution of the present invention uses a learning rate of 0.0002, 200 iterations, and 256 training batches for the training generator. When implanting a backdoor into the target model, the model is trained with a unified training round of 200 times and a training batch of 256. In the comparative experiment, an experiment with a training batch of 512 was also conducted. In the experiments with CIFAR10 and STL10 as downstream tasks, the target class is set to "airplane"; in the experiments with GTSRB as downstream tasks, the target class is set to "stop"; in the experiments with SVHN as downstream tasks, the target class is set to "zero".

[0142] 2. Poisoning data quality assessment

[0143] (1) Quality of generated poisoned data

[0144] The poisoned image contains invisible triggers. In order to evaluate the poisoned image generated by the technical solution of the present invention, it is compared with the current latest solutions BadEncoder and GhostEncoder backdoor attack framework. The experimental results are as follows: Figure 3 As shown. By naked eye observation, the poisoned image generated by the BadEncoder backdoor attack has an obvious white square in the lower right corner, while the poisoned image generated by the technical solution of the present invention and the poisoned image generated by the GhostEncoder backdoor attack framework can achieve high invisibility. This is because in the technical solution of the present invention, the two structures of network, UNet and PatchGAN, are suitable for image-to-image generative tasks. The discriminator PatchGAN structure can further enhance the ability of GAN to generate high-quality images. It divides the poisoned image input to the discriminator into 256 equal blocks, so that the discriminator can analyze the image features more carefully and ensure the high invisibility of the trigger.

[0145] (2) Quantitative analysis of backdoor concealment

[0146] In order to further quantitatively analyze the concealment of the backdoor, three indicators, Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS), are used to quantitatively analyze the concealment of the trigger in the poisoned image. Figure 4 shown.

[0147] from Figure 4As can be seen in (a), the technical solution SSL-BIST of the present invention achieves the highest PSNR value, and the BadEncoder framework has the lowest PSNR value. PSNR is defined based on the mean square error, and the distortion between the original image and the poisoned image can be measured to evaluate the quality of the poisoned image. The larger the PSNR value, the higher the backdoor concealment.

[0148] from Figure 4 As can be seen from (b), the technical solution SSL-BIST of the present invention achieves the highest SSIM value, and the BadEncoder framework has the lowest SSIM value. SSIM is used to measure the structural similarity between the original image and the poisoned image, and its value range is between [-1,1]. The larger the SSIM value, the higher the structural similarity between the poisoned image and the original image.

[0149] from Figure 4 As can be seen from (c), the LPIPS value of the technical solution SSL-BIST of the present invention is the lowest, and the LPIPS value of the BadEncoder framework is the highest. LPIPS is an image evaluation index designed based on the visual system of human perception. The experiment uses the vgg model to evaluate the LPIPS value. The smaller the LPIPS value, the higher the visual similarity between the poisoned image and the original image.

[0150] Depend on Figure 4 And the analysis shows that compared with the latest solutions, the poisoned image generated by the technical solution of the present invention has the highest quality. This is because the technical solution of the present invention converts the extracted image edge structure into the form of invisible disturbance, deeply integrates it with the image structure, and protects the consistency of the poisoned image and the clean image structure to the greatest extent while ensuring that the trigger is invisible.

[0151] 2. Comparative experiment

[0152] Jia et al. first proposed the BadEncoder framework for self-supervised learning backdoor attacks. They embedded a white square in the lower right corner of the training image as a trigger for the poisoned image, and then used the contrastive learning algorithm to indirectly construct a malicious association between the poisoned image and the target class to achieve backdoor implantation. In order to improve the concealment of the backdoor, Wang et al. designed the GhostEncoder backdoor attack framework. They used a pre-trained watermark encoder to steganographically write a specific string into the image to generate a poisoned image. The experiment in this section is compared with the latest research on self-supervised learning backdoor attacks mentioned above, and the attack capability and practicality of the technical solution of the present invention and these two backdoor attack strategies are analyzed.

[0153] (1) Attack capability

[0154] 1) Performance of SSL-BIST in ResNet-18

[0155] The experiment first used CIFAR10 and STL10 to train ResNet-18 from scratch as pre-training models, simulating the clean pre-training models obtained by attackers from public platforms. The experimental settings and the performance of the ResNet-18 pre-training model are shown in Table 1. In Table 1, under the premise of keeping the pre-training data, learning rate, and number of iterations unchanged, increasing the learning batch can improve the accuracy of the pre-training model.

[0156] Table 1 ResNet-18 pre-training

[0157]

[0158] This section generates poisoned data according to the methods of the SSL-BIST, BadEncoder, and GhostEncoder backdoor attack frameworks, sets the learning rate to 0.001, uses the same number of iterations 200 and sample batches 256 to train the target model ResNet-18, and implants the backdoor into it. The experiment sets different target classes in different downstream task datasets. The target class for downstream tasks CIFAR10 and STL10 is set to "airplane", the target class for downstream tasks GTSRB is set to "stop", and the target class for downstream tasks SVHN is set to "zero". Backdoors are implanted in the ResNet-18 pre-trained models trained for CIFAR10 and STL10. The evaluation results are as follows: Figure 5 and Figure 6 shown.

[0159] Figure 5 (a) and Figure 6 In (a), in the backdoor implantation task with STL10, CIFAR10, GTSRB, and SVHN as downstream datasets, BadEncoder and SSL-BIST can achieve a backdoor attack success rate close to 100%. GhostEncoder can achieve a high attack success rate in the STL10 task, but performs poorly in GTSRB and SVHN. This shows that although only stego-writing fixed strings into images can greatly improve the concealment of backdoors, the features of the trigger are difficult to be recognized by the model, resulting in a low attack success rate.

[0160] Figure 5 (b) and Figure 6In (b), in the backdoor implantation task with STL10, CIFAR10, GTSRB and SVHN as downstream data sets, the false trigger rate of SSL-BIST of the technical solution of the present invention is lower than that of BadEncoder and GhostEncoder, and the false trigger rate of GhostEncoder is lower than that of BadEncoder. This shows that invisible triggers are more advantageous in reducing the false trigger rate of backdoor attacks. The poisoned data generated by the technical solution of the present invention can significantly reduce the false trigger rate while maintaining a high success rate of backdoor attacks.

[0161] Figure 5 (c)(d) and Figure 6 In (c) and (d), SSL-BIST, BadEncoder, and GhostEncoder can maintain the accuracy of the backdoor model at a comparable level to that of the clean model. It is worth noting that the ACC index with SVHN as the downstream task is the lowest. This is because the SVHN dataset has more noise and the data of various types are unevenly distributed. And because the backdoor model generates more similar feature vectors for the poisoned data, the experimental results show that B-ACC is generally higher than ACC.

[0162] 2) Performance with larger learning batches

[0163] This section verifies the performance of SSL-BIST under larger learning batches through experiments. Two sets of experiments are designed. The experiments use the ResNet-18 pre-trained models trained with CIFAR10 and STL10 respectively. The learning batch of the control group is set to 256, the learning rate is set to 0.001, and the number of iterations is set to 200. The learning batch of the experimental group is set to 512, the learning rate is set to 0.001, and the number of iterations is set to 200. The evaluation results are shown in Figure 2. Figure 7 and Figure 8 shown.

[0164] exist Figure 7 (a) and Figure 8 In (a), the backdoor attack success rate ASR of the learning batch of 512 is significantly higher than that of the control group of the learning batch of 256. Figure 7 (b) with Figure 8 In (b), the false trigger rate FAR is lower when the learning batch is 512, indicating that increasing the learning batch can effectively improve the attack performance and backdoor concealment of SSL-BIST. This is because after increasing the learning batch, more samples can be processed in each iteration, which improves the stability of gradient estimation. At the same time, when calculating the gradient, the noise caused by a single sample is averaged, which helps to reduce the noise in the training process and improve the performance of backdoor attacks.

[0165] (2) Practicality

[0166] This section applies SSL-BIST to more fields and verifies the practicality of the technical solution of the present invention in more scenarios through experiments. Since the GhostEncoder backdoor attack framework cannot be applied to ResNet-18, this section only compares SSL-BIST with the BadEncoder backdoor attack framework.

[0167] 1) Application of SSL-BIST

[0168] Compared with ResNet-18, ResNet-50 has more convolutional layers and a more complex structure. We use it as the target model to verify the performance of SSL-BIST and BadEncoder. This section downloads the ResNet-50 pre-trained model trained on ImageNet from the open source platform as the target model. The experiment sets the learning rate to 0.0001, the number of iterations to 200, the learning batch to 256, and the downstream task datasets to STL10, GTSRB, and SVHN. The experimental results are shown in Figure 2. Fig. 9 shown.

[0169] exist Fig. 9 In (a), the backdoor attack success rate of SSL-BIST ASR exceeds that of BadEncoder only in the dataset with downstream task GTSRB, but SSL-BIST also maintains the attack success rate at around 99%, which is at the same level as the performance of BadEncoder. Fig. 9 In (b), the false trigger rate FAR of SSL-BIST is significantly lower than that of BadEncoder, indicating that the technical solution of the present invention has higher backdoor concealment. Fig. 9 (b)(c) It can be seen that although SSL-BIST and BadEncoder maintain the same level of clean model accuracy ACC, the backdoor model accuracy B-ACC of SSL-BIST is lower than that of BadEncoder. This is because the poisoned data generated by SSL-BIST is very different from the clean data, which causes the attacker to reduce the similarity between the clean data of the target class in the downstream dataset when using a more complex pre-trained model for downstream task training, resulting in a decrease in the accuracy of the backdoor model. In actual situations, if B-ACC needs to be improved, the quality of the poisoned data and the B-ACC indicator can be balanced according to the actual situation by reducing the number of GAN training rounds for generating triggers or using a smaller dataset for training.

[0170] Furthermore, the performance of the technical solution of the present invention is further verified on the CLIP model. The CLIP pre-trained model trained with 300 million unlabeled image-text pairs is downloaded from the OpenAI platform as the target model. The learning rate is set to 0.0001, the number of iterations is set to 200, the learning batch is set to 256, and the downstream task data sets are STL10, GTSRB, and SVHN. The experimental results are as follows: Fig.10 shown.

[0171] exist Fig.10 In (a), the backdoor attack success rate ASR of SSL-BIST is higher than that of BadEncoder, which shows that in the multimodal model, the technical solution of the present invention can achieve higher attack performance. Fig.10 In (b), the false trigger rate of SSL-BIST is significantly lower than that of BadEncoder, and is even close to 0 in STL10 and GTSRB. This shows that in multimodal models with stronger feature extraction capabilities, the characteristics of image structure triggers have almost no impact on the performance of clean models, and have excellent backdoor concealment. Fig.10 It can be seen from (b) and (c) that the technical solution of the present invention can be consistent with the current latest solution in terms of clean model accuracy ACC and backdoor model accuracy B-ACC.

[0172] 2) Optimal range of hyperparameters

[0173] In this section, we continue to explore the impact of the values ​​of hyperparameters α and β on ASR and B-ACC through the control variable method, and determine the optimal value range of α and β. The experiment uses the CIFAR10 dataset to train ResNet-18 from scratch as a pre-training model, with the learning rate set to 0.001 and the learning batch set to 512. The downstream tasks are trained using the GTSRB training set, with the learning rate set to 0.0001 and the learning batch set to 512. The results are shown in Figure 2. Fig.11 shown.

[0174] exist Fig.11 In (a), when α = 0, that is, the attack effectiveness loss is removed, the attack success rate ASR and the backdoor model accuracy B-ACC are both 0. Fig.11 In (a), when β = 0, that is, the attack concealment loss is removed, SSL-BIST also loses almost all performance. Fig.11 The curve results of (a) and (b) show that whether removing the loss of attack effectiveness or the loss of attack concealment, it will affect ASR and B-ACC and reduce the performance of SSL-BIST. When α and β are both set to values ​​greater than 1, ASR and B-ACC can be stabilized in a relatively high range, achieving a higher attack success rate and higher accuracy. This shows that the SSL-BIST framework can effectively implement the implantation of backdoors.

[0175] 3) Impact of data transformation on backdoor activation

[0176] In order to further explore the practicality of poisoned data in actual situations, the experiment rotates and deforms the poisoned data, and uses the transformed poisoned data to activate the backdoor in the CLIP model. Taking the use of the GTSRB dataset to train downstream tasks as an example, the performance of the technical solution of the present invention and BadEncoder after data processing is compared. Fig.12 The experimental results are shown in Fig.13 shown.

[0177] exist Fig.12 In the example, the poisoned data generated by SSL-BIST and BadEncoder are processed by rotating 90 degrees clockwise, rotating 180 degrees clockwise, and deforming the data. Fig.13 It can be seen that after the transformation, the attack success rate ASR of SSL-BIST only slightly decreases after the data deformation processing, while BadEncoder almost loses its attack performance after three types of data processing. This is because the technical solution of the present invention hides the trigger in the image structure during the stage of generating poisoned data. The image structure can effectively protect the semantic features of the trigger activating the backdoor, so that the poisoned image will not lose the function of activating the backdoor after rotation. At the same time, the trigger information of activating the backdoor is dispersed throughout the image, so that the poisoned data can still retain sufficient trigger information after deformation processing. At the same time, this also shows that the technical solution of the present invention is more practical.

[0178] Example 2

[0179] A system using the self-supervised learning backdoor attack method based on image structure triggers, comprising:

[0180] Image edge extraction construction unit: extracts the image edge by using the image edge extraction method, colors it using the RGB channel color values, and obtains the trigger of the corresponding image;

[0181] Poisoned image construction unit: Through the poisoned image generation adversarial network, the trigger is converted into an invisible perturbation and hidden in the edge structure of the image. The adversarial generation network of the poisoned image uses UNet, the discriminator uses PatchGAN, and an auxiliary guidance network is added to ensure that the target model can extract the features of the trigger in the poisoned image.

[0182] Backdoor implantation unit: By using SimCLR comparative learning, a concealment loss function and two concealment loss functions are constructed. The final loss function is a linear combination of the effectiveness loss function and the concealment loss function. By gradually increasing the cosine similarity between the poisoned data and the clean data of the target class, the malicious association between the poisoned data and the target class is indirectly constructed.

[0183] Example 3

[0184] A computer-readable storage medium stores a computer program, wherein the computer program is called by a processor to execute:

[0185] The above steps are a multimodal invisible backdoor attack method based on adversarial perturbation.

[0186] For the specific implementation process of each step, please refer to the description of the above method.

[0187] The readable storage medium is a computer-readable storage medium, which may be an internal storage unit of the controller described in any of the foregoing embodiments, such as a hard disk or memory of the controller. The readable storage medium may also be an external storage device of the controller, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the controller. Further, the readable storage medium may also include both an internal storage unit of the controller and an external storage device. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0188] Example 4

[0189] An electronic terminal, comprising at least:

[0190] one or more processors;

[0191] one or more memories;

[0192] The processor calls the computer program stored in the memory to execute:

[0193] The above steps are a multimodal invisible backdoor attack method based on adversarial perturbation.

[0194] For the specific implementation process of each step, please refer to the description of the above method.

[0195] Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned readable storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk, etc. Various media that can store program codes.

[0196] It should be emphasized that the examples described in the present invention are illustrative rather than restrictive, and therefore the present invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solution of the present invention that do not depart from the purpose and scope of the present invention, whether modified or replaced, also fall within the scope of protection of the present invention.

Claims

1. A self-supervised learning backdoor attack method based on image structure trigger, characterized in that: include: Construct poisoned images: Use the RCF scheme to extract the edges of clean images and color them as the input of the poisoned data generation adversarial model. Use UNet as the generator and PatchGAN as the discriminator, and add an auxiliary guidance network after the generation model to guide the generation model to learn trigger features. Finally, only the generator is retained as the poisoned data generation model; Constructing a loss function for implanting a backdoor: Use SimCLR contrastive learning in self-supervised learning to design a loss function for implanting a backdoor, including one validity loss and two concealment losses. In the process of minimizing the validity loss, the feature vectors generated by the backdoor model for the poisoned data and the clean data of the target class become more and more similar. In the process of minimizing the concealment loss, the feature vectors generated by the clean model and the backdoor model for the clean data become more and more similar; Training the target model: Use the constructed poisoned images and loss function to control the training process. During the training process, the cosine similarity between the poisoned images and the clean images of the target class is continuously improved, indirectly building a malicious association between the trigger and the target class, while maintaining the accuracy of the backdoor model in recognizing clean images. By controlling the training process, the attacker can implant a backdoor into the pre-trained model or implant a backdoor during downstream tasks. This method is used to implant a backdoor into the target model ResNet-18, ResNet-50, and the multimodal CLIP model.

2. The method according to claim 1, characterized in that The original trigger is generated by the following formula: in Indicates the color filling operation, t i is a trigger, (r, g, b) can take any value, retaining the diversity of trigger design. An important component in the poisoned data generation model is the poisoned image generator f G (), we use it to generate poisoned images 3. The method according to claim 2, characterized in that The poisoned images are generated by generating adversarial networks, where the generator is a UNet structure, the discriminator is a PatchGAN structure, and the objective functions of the generator and discriminator are: Among them, L G is the generator objective function, L D is the discriminator objective function; x i It is clean data.

4. The method according to claim 3, characterized in that In order to make the trigger features in the poisoned image recognizable by the target model, we add an auxiliary guidance network after the generator to guide the generation network to learn the features of the trigger and ensure the trigger effect of the poisoned image. The loss function of the auxiliary guidance network is: Among them, x c is the mapping of the clean image, X and X * Represent the clean dataset and poisoned dataset respectively.

5. The method according to claim 1, characterized in that The loss function of the backdoor implantation includes an effectiveness loss function and two concealment loss functions; The effectiveness loss function is: The two hidden loss functions are: Where sim(,) is an abstract function for solving the cosine similarity of two feature vectors, |D| is the amount of training data selected by the attacker for implanting the backdoor, |D i | is the i-th type of test data used to test the performance of the model, n i is the amount of test data, and each test class contains m test data.

6. The method according to claim 5, characterized in that The final loss function of the implanted backdoor is: L=L eff +α·L con1 +β·L con2 Here α and β are hyperparameters, α controls the downstream task test dataset D i Impact on model accuracy, β controls the impact of the dataset D used to implant the backdoor on the model accuracy. By default, α=β=1.

7. A self-supervised learning backdoor attack system based on image structure trigger according to any one of claims 1 to 6, characterized in that: include: Image edge extraction construction unit: extracts the image edge by using the image edge extraction method, colors it using the RGB channel color values, and obtains the trigger of the corresponding image; Poisoned image construction unit: Through the poisoned image generation adversarial network, the trigger is converted into an invisible perturbation and hidden in the edge structure of the image. The adversarial generation network of the poisoned image uses UNet, the discriminator uses PatchGAN, and an auxiliary guidance network is added to ensure that the target model can extract the features of the trigger in the poisoned image. Backdoor implantation unit: By using SimCLR comparative learning, a concealment loss function and two concealment loss functions are constructed. The final loss function is a linear combination of the effectiveness loss function and the concealment loss function. By gradually increasing the cosine similarity between the poisoned data and the clean data of the target class, the malicious association between the poisoned data and the target class is indirectly constructed.

8. An electronic device, characterized in that: At least: one or more processors; a memory storing one or more computer programs; The processor calls a computer program to execute: The steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: A computer program is stored, which is called by a processor to execute: The steps of the method according to any one of claims 1 to 6.