Task execution method for self-distillation model back door removal, electronic equipment and medium

Abstract

The invention discloses a self-distillation model backdoor removal task execution method, electronic equipment and a medium, and the method comprises the steps: obtaining an original task data set, and removing labels corresponding to a part of samples; performing data amplification on the labeled data; respectively performing first and second data enhancement operations on the unlabeled data to obtain a first unlabeled data set and a second unlabeled data set; constructing positive and negative sample pairs; respectively inputting the first and second label-free data sets into the first and second feature extraction models; constructing a comparison loss and self-distillation feature map loss to train a first feature extraction model, and gradually increasing the distance between the feature vectors corresponding to the negative samples and gradually reducing the distance between the feature vectors corresponding to the positive samples by the comparison loss number; updating the second feature extraction model through the momentum; inputting the amplified labeled data into the trained first feature extraction model, and constructing cross entropy loss to re-train the first feature extraction model; and executing a task by using the trained first feature extraction model.

Description

Technical Field

[0001] The present invention belongs to the technical field of model backdoor removal, and in particular relates to a task execution method, electronic equipment, and medium for self-distillation model backdoor removal. Background Art

[0002] In recent years, deep learning technology has experienced rapid development and has been widely used in many fields, such as image recognition, natural language processing, speech recognition, etc. Since training a model from scratch requires a lot of computing resources, it has become increasingly popular to obtain pre-trained model backbones from third-party platforms and use them for various downstream tasks. However, this also brings many security issues, among which the most prominent security vulnerability is backdoor attack.

[0003] Among them, the targets of backdoor attacks are:

[0004] (1) The backdoor model has normal accuracy on clean test samples;

[0005] (2) The backdoor model will produce the prediction results pre-specified by the attacker if and only if the test sample contains a pre-set backdoor trigger.

[0006] Among them, goal (1) ensures the concealment of backdoor attacks, and goal (2) ensures that the backdoor model can be arbitrarily manipulated by the attacker. The attacker inserts a backdoor trigger into the target model in some way before or during training, causing the model to produce incorrect or manipulated outputs when a specific input is provided during testing, which may have terrible consequences in real applications. For example, a traffic sign recognition system using a backdoor model may always misclassify the "STOP" sign as "GO STRAIGHT" when a specific pattern appears, resulting in serious security issues. Therefore, efficient defense against backdoor attacks is an extremely challenging task. At present, there are three main defense strategies against backdoor attacks: backdoor model detection, backdoor sample detection, and backdoor removal. Among them, backdoor removal is the most critical.

[0007] The main goal of backdoor removal is to remove the backdoor from the backdoor model while maintaining the normal performance of the model. Backdoor removal is crucial to backdoor defense and can play an important role in actual application scenarios, so most of the work of backdoor defense is centered around backdoor removal.

[0008] Existing backdoor removal methods can be roughly divided into two categories:

[0009] (1) Removal during training: Detect potential backdoor samples during model training and prevent the model from learning these samples;

[0010] (2) Removal after training: remove the implanted backdoor triggers from the backdoor model to restore the pure functionality of the model.

[0011] At present, the main methods for backdoor removal include fine pruning, model attention distillation and other methods. Model fine pruning integrates the two technologies of pruning and fine-tuning, which can effectively eliminate the backdoor in the model. It first uses pruning to cut out the backdoor neurons associated with the trigger from the backdoor model, thereby achieving the effect of model purification. Then fine-tune with a small amount of clean data to restore the performance of the model on clean samples. Model attention distillation removes backdoor triggers by using knowledge distillation and a small amount of clean data. Because knowledge distillation involves two models (i.e., teacher model and student model), backdoor defense based on knowledge distillation needs to select appropriate teacher model and student model. The core of this method is to find a suitable attention representation to ensure the effectiveness of distillation defense.

[0012] However, the existing more advanced neural attention distillation (NAD) and adversarial neuron pruning (ANP) cannot resist attacks like ATTEQ-NN. Moreover, the existing backdoor removal process not only cannot repair the damage to model performance caused by backdoor injection, but further reduces model performance, hindering the further application of backdoor removal methods. In addition, most existing methods perform distillation to remove backdoors when there is a lot of clean data and more complete labels. It is impossible to achieve effective backdoor removal in scenarios with only a small amount of clean data and missing labels. Summary of the invention

[0013] In view of the shortcomings of the prior art, the present invention provides a task execution method, electronic device, and medium for removing the backdoor of a self-distillation model.

[0014] In a first aspect, an embodiment of the present invention provides a method for executing a task for removing a backdoor from a distillation model, the method comprising:

[0015] Get the original task dataset and remove the labels corresponding to some samples in the original task dataset;

[0016] Perform data augmentation on the labeled data in the original task dataset;

[0017] Performing a first data enhancement operation and a second data enhancement operation on the unlabeled data in the original task data set to obtain a first unlabeled data set and a second unlabeled data set;

[0018] Two samples obtained after a first data enhancement operation and a second data enhancement operation on an unlabeled sample are used as a positive sample pair; a sample obtained after a data enhancement operation on an unlabeled sample and a sample obtained after a data enhancement operation on another unlabeled sample are used as a negative sample pair;

[0019] Constructing a first feature extraction model and a second feature extraction model identical thereto; inputting a first unlabeled data set into the first feature extraction model, and inputting a second unlabeled data set into the second feature extraction model; constructing contrast loss and self-distillation feature map loss to train the first feature extraction model, wherein the contrast loss number is used to minimize the distance between feature vectors corresponding to positive samples and maximize the distance between feature vectors corresponding to negative samples; updating the second feature extraction model through momentum;

[0020] The amplified labeled data is input into the trained first feature extraction model, and the cross entropy loss is constructed to train the first feature extraction model again;

[0021] Use the trained first feature extraction model to perform the task.

[0022] Furthermore, the process of performing a first data enhancement operation and a second data enhancement operation on the unlabeled data in the original task data set to obtain a first unlabeled data set and a second unlabeled data set includes:

[0023] Performing a first data enhancement operation on the unlabeled data in the original task data set to obtain a first unlabeled data set; performing a second data enhancement operation on the unlabeled data in the original task data set to obtain a second unlabeled data set;

[0024] The first data enhancement operation is different from the second data enhancement operation.

[0025] Furthermore, the first data enhancement operation and the second data enhancement operation are selected from flipping, rotating, scaling, random cropping, color distortion or color adjustment.

[0026] Furthermore, the process of constructing contrast loss and self-distillation feature map loss to train the first feature extraction model includes:

[0027] The expression of contrast loss is as follows:

[0028]

[0029] Where N is the number of samples in the first unlabeled dataset, q i is the feature vector obtained by the first feature extraction model for the input sample i, is the feature vector obtained by the second feature extraction model as the positive sample of input sample i, is the feature vector obtained by the second feature extraction model as the negative sample of input sample i, τ is the temperature coefficient;

[0030] The expression of the self-distillation feature map loss is as follows:

[0031]

[0032] In the formula, F i is the output feature of the i-th feature layer, F D is the output feature of the deepest feature layer, Represents the sum of the p-th power of the absolute value of the output feature.

[0033] Furthermore, the process of updating the second feature extraction model by momentum includes:

[0034] The expression for momentum update is as follows:

[0035] θ k ←mθ k +(1-m)θ q

[0036] In the formula, θ k is the parameter of the second feature extraction model, θ q is the parameter of the main encoder, the parameter θ of the main encoder q Updates are back-propagated based on the contrastive loss, where m is the momentum coefficient.

[0037] Furthermore, the process of constructing the contrast loss and the self-distillation feature map loss to train the first feature extraction model also includes:

[0038] Add a fully connected layer after each feature layer of the first feature extraction model so that the output dimension of each feature layer is the same as the output dimension of the first feature extraction model;

[0039] The process of inputting the amplified labeled data into the trained first feature extraction model also includes:

[0040] A fully connected network is added after the first feature extraction model to reduce the dimension of the extracted features to the confidence dimension of the classification.

[0041] Further, constructing a cross entropy loss to retrain the first feature extraction model includes:

[0042] The expression of cross entropy loss is as follows:

[0043]

[0044] Where, L i represents the output of the i-th feature layer, q i represents the confidence output of the i-th feature layer, y represents the final softmax output of the first feature extraction model, C represents the number of categories, j represents the j-th category, j∈C, q ij Represents the confidence output of the i-th feature layer on the j-th category.

[0045] In a second aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned task execution method for removing the backdoor of the self-distillation model.

[0046] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the task execution method for removing the backdoor of the self-distillation model.

[0047] In a fourth aspect, an embodiment of the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the above-mentioned task execution method for removing the backdoor of the self-distillation model.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The present invention performs feature self-distillation on the first feature model on unlabeled data through comparative learning, and performs label self-distillation on a small amount of labeled data, thereby removing the backdoor of the entire network and ensuring the main task performance of the model. The first feature model trained by the present invention greatly reduces the success rate of backdoor attacks while maintaining its main task performance, and has a good defense effect against various backdoor attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0051] Figure 1 A flowchart of a task execution method for removing a backdoor from a self-distillation model provided by an embodiment of the present invention;

[0052] Figure 2 A flowchart of a task execution method for removing a backdoor from a self-distillation model provided by an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] It should be noted that, in the absence of conflict, the features in the following embodiments and implementations may be combined with each other.

[0056] like Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a task execution method for removing a backdoor from a distillation model, the method comprising the following steps:

[0057] Step S1, obtaining the original task data set, and removing the labels corresponding to some samples in the original task data set;

[0058] Perform data augmentation on the labeled data in the original task dataset;

[0059] Performing a first data enhancement operation and a second data enhancement operation on the unlabeled data in the original task data set to obtain a first unlabeled data set and a second unlabeled data set;

[0060] Two samples obtained after a first data enhancement operation and a second data enhancement operation on an unlabeled sample are used as a positive sample pair; a sample obtained after a data enhancement on an unlabeled sample and a sample obtained after a data enhancement on another unlabeled sample are used as a negative sample pair.

[0061] Specifically, step S1 includes the following sub-steps:

[0062] Step S101, download and decompress the original task data set, and correspond each image sample in the original task data set to its label one by one.

[0063] It should be noted that the original task dataset can use multimodal (image, text, voice, etc.) data to perform different downstream tasks (image classification, object detection, semantic segmentation, etc.).

[0064] Step S102, setting a first ratio, dividing the original task data set into a training data set and a test data set according to the first ratio; setting a second ratio, removing labels corresponding to samples in the training data set according to the second ratio.

[0065] Step S103, performing data augmentation on the labeled data in the training data set; performing a first data augmentation operation and a second data augmentation operation on the unlabeled data in the original task data set to obtain a first unlabeled data set and a second unlabeled data set, and constructing positive sample pairs and negative samples.

[0066] Furthermore, operations including translation, rotation, and scaling are performed on the labeled data in the training dataset to obtain data that is 4 times larger than the original data, which is used for subsequent model label self-distillation and performance fine-tuning.

[0067] A first data enhancement operation is performed on the unlabeled data in the original task data set to obtain a first unlabeled data set; a second data enhancement operation is performed on the unlabeled data in the original task data set to obtain a second unlabeled data set; wherein the first data enhancement operation is different from the second data enhancement operation.

[0068] Among them, the process of performing data enhancement on the unlabeled data in the training data set to obtain positive sample pairs and negative sample pairs includes:

[0069] The two samples obtained after data enhancement of an unlabeled sample in the training data set are used as positive sample pairs; one sample obtained after data enhancement of an unlabeled sample in the traffic sign training data set and one sample obtained after data enhancement of another unlabeled sample are used as negative sample pairs; thus, the preprocessing operation of the original task data set is completed.

[0070] For example, an unlabeled sample A in the training data set is augmented (the two samples A obtained after the operation 1 , A 2 , A 1 , A 2 As a positive sample pair (A 1 ,A 2 ) ; Two samples B are obtained by data augmentation operation on an unlabeled sample B in the training data set 1 , B 2 , will (A 1 ,B 1 )、(A 1 ,B 2 )、(A 2 ,B 1 )、(A 2 ,B 2 ) as negative sample pairs.

[0071] Step S2, constructing a first feature extraction model and a second feature extraction model identical thereto; inputting the first unlabeled data set into the first feature extraction model, and inputting the second unlabeled data set into the second feature extraction model; constructing contrast loss and self-distillation feature map loss to train the first feature extraction model, wherein the contrast loss number is used to minimize the distance between feature vectors corresponding to positive samples and maximize the distance between feature vectors corresponding to negative samples; updating the second feature extraction model through momentum.

[0072] Add a fully connected layer after each feature layer of the first feature extraction model so that the output dimension of each feature layer is the same as the output dimension of the first feature extraction model;

[0073] Furthermore, the expression of the contrast loss (InfoNCE Loss, Noise Contrastive Estimation Loss) is as follows:

[0074]

[0075] Where N is the number of samples in the first unlabeled dataset, q i is the feature vector obtained by the first feature extraction model for the input sample i, is the feature vector obtained by the second feature extraction model as the positive sample of input sample i, is the feature vector obtained by the second feature extraction model as the negative sample of input sample i; τ is the temperature coefficient, which is used to adjust the distribution of similarity scores and to control the model's discrimination of negative samples.

[0076] It should be noted that if the samples input into the first feature extraction model and the second feature extraction model are positive sample pairs, then we hope that their features are as close as possible, that is, the similarity label is 1; if they are negative sample pairs, then we hope that their features are as far apart as possible, that is, the similarity label is 0.

[0077] Furthermore, the expression of the self-distillation feature map loss is as follows:

[0078]

[0079] In the formula, F i is the output feature of the i-th feature layer, F D is the output feature of the deepest feature layer, Represents the sum of the p-th power of the absolute value of the output feature.

[0080] It should be noted that the self-distillation feature map loss is the loss between the output features of each shallow feature layer of the model and the output features of the deepest feature layer. By minimizing the difference between the shallow features and the output features, the first feature extraction model can achieve self-distillation of features from the deepest layer to the shallowest layer, thereby realizing the backdoor removal of the entire first feature extraction model at the output feature level.

[0081] Furthermore, the expression for momentum update is as follows:

[0082] θ k ←mθ k +(1-m)θ q

[0083] In the formula, θ k is the parameter of the second feature extraction model, θ q is the parameter of the main encoder, the parameter θ of the main encoder q Based on the contrastive loss back-propagation update, m is the momentum coefficient, which usually ranges from 0.99 to 1 and is used to control the update speed of the momentum encoder.

[0084] It should be noted that updating the second feature extraction network at a slower rate through momentum means that its update is more stable, thereby ensuring that during the contrastive learning process, the feature vector of the negative sample will not change significantly due to the drastic update of the first feature extraction network, thus avoiding the loss of feature consistency due to the drastic changes in the first feature extraction network, while also keeping the second feature extraction network in an updated state.

[0085] Step S3, input the amplified labeled data into the trained first feature extraction model, construct a cross entropy loss and train the first feature extraction model again.

[0086] A fully connected network is added after the first feature extraction model to reduce the dimension of the extracted features to the confidence dimension of the classification.

[0087] Furthermore, the expression of cross entropy loss is as follows:

[0088]

[0089] Where, L i represents the output of the i-th feature layer, q i represents the confidence output of the i-th feature layer, y represents the final softmax output of the first feature extraction model, C represents the number of categories, j represents the j-th category, j∈C, q ij Represents the confidence output of the i-th feature layer on the j-th category.

[0090] It should be noted that the difference between the output confidence of the deepest layer and the output confidence of other feature layers is minimized, and the cross entropy loss is used to achieve label self-distillation. While improving the performance of the main task, the backdoor is completely removed at the output confidence level. The fully connected layer added to the feature layer is only used for label self-distillation, and then deleted after the backdoor is removed. The knowledge hidden in the labeled dataset is directly introduced into the classifiers of all feature layers through labels. As the network develops from shallow to deep layers, the weight assigned to the classifier gradually increases, realizing the removal of the model backdoor.

[0091] Step S4, using the trained first feature extraction model to perform the task.

[0092] Example 1

[0093] This embodiment uses the CTSDB traffic sign dataset as the original task dataset and uses the identification and classification of traffic signs as the downstream task, so as to further illustrate the specific implementation process of the task execution method for removing the backdoor of the self-distillation model provided by the present invention, specifically, including the following steps:

[0094] Step S1, obtain the CTSDB traffic sign dataset, and remove the labels corresponding to the samples in the CTSDB traffic sign dataset in proportion; perform data amplification on the labeled data in the CTSDB traffic sign dataset; perform data enhancement on the unlabeled data in the CTSDB traffic sign dataset to obtain a positive sample pair dataset and a negative sample pair dataset.

[0095] Specifically, step S1 includes the following sub-steps:

[0096] Step S101, download and decompress the CTSDB traffic sign dataset, and correspond each image sample in the CTSDB traffic sign dataset to its label one by one.

[0097] Specifically, in this example, the China Traffic Sign Database (CTSDB) is selected as the original task dataset. The CTSDB traffic sign dataset is a dataset specially created for traffic sign recognition and classification tasks, and is mainly used for research in the fields of autonomous driving and intelligent transportation systems. The CTSDB traffic sign dataset contains a large number of Chinese traffic sign images, covering various types of traffic signs, such as warning signs, prohibition signs, and instruction signs.

[0098] Step S102, setting a first ratio, dividing the CTSDB traffic sign dataset into a traffic sign training dataset and a traffic sign test dataset according to the first ratio; setting a second ratio, removing labels corresponding to samples in the traffic sign training dataset according to the second ratio.

[0099] Specifically, in this example, 10,000 images are randomly selected as the traffic sign training data set and 2,000 images are randomly selected as the traffic sign test data set, and the labels of the traffic sign training data set are removed at a ratio of 7:3, so that the traffic sign training data set obtains a large amount of unlabeled data and a small amount of labeled data.

[0100] Step S103, performing data augmentation on the labeled data in the traffic sign training data set; performing a first data augmentation operation and a second data augmentation operation on the unlabeled data in the traffic sign training data set to obtain a first unlabeled data set and a second unlabeled data set, and constructing positive sample pairs and negative samples.

[0101] Furthermore, in this example, the process of performing data augmentation on the labeled data in the traffic sign training data set includes:

[0102] The labeled data in the traffic sign training dataset are subjected to operations including translation, rotation, and scaling to obtain data that is 4 times larger than the original data, which is used for subsequent model label self-distillation and performance fine-tuning.

[0103] Furthermore, in this example, a first data enhancement operation is performed on the unlabeled data in the traffic sign training data set (in this example, the first data enhancement operation uses random cropping) to obtain a first unlabeled data set; and a second data enhancement operation is performed on the unlabeled data in the original task data set (in this example, the first data enhancement operation uses color distortion) to obtain a second unlabeled data set.

[0104] Among them, the process of performing data enhancement on the unlabeled data in the traffic sign training data set to obtain positive sample pairs and negative sample pairs includes:

[0105] The two samples obtained by random cropping and color distortion of an unlabeled sample in the traffic sign training dataset are used as positive sample pairs; the sample obtained by random cropping and color distortion of an unlabeled sample in the traffic sign training dataset and the sample obtained by random cropping and color distortion of another unlabeled sample are used as negative sample pairs; the preprocessing operation of the CTSDB traffic sign dataset is completed.

[0106] For example, the unlabeled samples {A, B, C, ...} in the traffic sign training dataset are randomly cropped to obtain {A 1 ,B 1 ,C 1 , …}, as the first unlabeled data set; the unlabeled samples {A, B, C, …} in the traffic sign training data set are color distorted to obtain {A2 ,B 2 ,C 2 ,……}, as the second unlabeled dataset.

[0107] For example, an unlabeled sample A in the traffic sign training data set is randomly cropped and color distorted to obtain two samples A 1 , A 2 , A 1 , A 2 As a positive sample pair (A 1 ,A 2 ) ; An unlabeled sample B in the traffic sign training data set is randomly cropped and color distorted to obtain two samples B 1 , B 2 , will (A 1 ,B 1 )、(A 1 ,B 2 )、(A 2 ,B 1 )、(A 2 ,B 2 ) as negative sample pairs.

[0108] Step S2, constructing a first feature extraction model and a second feature extraction model identical thereto; inputting the first unlabeled data set into the first feature extraction model, and inputting the second unlabeled data set into the second feature extraction model; constructing contrast loss and self-distillation feature map loss to train the first feature extraction model, wherein the contrast loss number is used to minimize the distance between feature vectors corresponding to positive samples and maximize the distance between feature vectors corresponding to negative samples; updating the second feature extraction model through momentum.

[0109] Step S3, input the amplified labeled data into the trained first feature extraction model, construct a cross entropy loss and train the first feature extraction model again.

[0110] Step S4, input the traffic sign test data set into the trained first feature extraction model to obtain the traffic sign recognition result. This example uses contrast learning self-distillation to effectively remove the backdoor in the case of a small amount of data and missing labels, and protect the performance of the road sign classifier.

[0111] In summary, the present invention performs feature self-distillation on the first feature model on unlabeled data through comparative learning, and performs label self-distillation on a small amount of labeled data, thereby achieving backdoor removal of the entire network while ensuring the main task performance of the model. The first feature model trained by the present invention greatly reduces the success rate of backdoor attacks while maintaining its main task performance, and has a good defense effect against various backdoor attacks.

[0112] Accordingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the task execution method for removing the backdoor of the self-distillation model as described above. Figure 3 As shown, a hardware structure diagram of any device with data processing capability in which the task execution method of removing the backdoor of the self-distillation model provided by the embodiment of the present invention is located, except Figure 3 In addition to the processor, memory and network interface shown, any device with data processing capability in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capability, which will not be described in detail.

[0113] Accordingly, the present application also provides a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by the processor, the task execution method for removing the backdoor of the self-distillation model as described above is implemented. The computer-readable storage medium can be an internal storage unit of any device with data processing capabilities described in any of the foregoing embodiments, such as a hard disk or a memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium can also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store data that has been output or is to be output.

[0114] Those skilled in the art will readily appreciate other embodiments of the present application after considering the description and practicing the contents disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application. The description and examples are intended to be exemplary only.

[0115] It should be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A task execution method for removing backdoors from a self-distillation model, characterized in that: The method comprises: Get the original task dataset and remove the labels corresponding to some samples in the original task dataset; Perform data augmentation on the labeled data in the original task dataset; Performing a first data enhancement operation and a second data enhancement operation on the unlabeled data in the original task data set to obtain a first unlabeled data set and a second unlabeled data set; Two samples obtained after a first data enhancement operation and a second data enhancement operation on an unlabeled sample are used as a positive sample pair; a sample obtained after a data enhancement operation on an unlabeled sample and a sample obtained after a data enhancement operation on another unlabeled sample are used as a negative sample pair; Constructing a first feature extraction model and a second feature extraction model identical thereto; inputting a first unlabeled data set into the first feature extraction model, and inputting a second unlabeled data set into the second feature extraction model; constructing contrast loss and self-distillation feature map loss to train the first feature extraction model, wherein the contrast loss number is used to minimize the distance between feature vectors corresponding to positive samples and maximize the distance between feature vectors corresponding to negative samples; updating the second feature extraction model through momentum; The amplified labeled data is input into the trained first feature extraction model, and the cross entropy loss is constructed to train the first feature extraction model again; Use the trained first feature extraction model to perform the task.

2. According to claim 1, a task execution method for removing backdoors from a self-distillation model is characterized in that: The process of performing a first data enhancement operation and a second data enhancement operation on the unlabeled data in the original task data set to obtain a first unlabeled data set and a second unlabeled data set includes: Performing a first data enhancement operation on the unlabeled data in the original task data set to obtain a first unlabeled data set; performing a second data enhancement operation on the unlabeled data in the original task data set to obtain a second unlabeled data set; The first data enhancement operation is different from the second data enhancement operation.

3. A task execution method for removing backdoors from a self-distillation model according to claim 1 or 2, characterized in that: The first data enhancement operation and the second data enhancement operation are selected from flipping, rotating, scaling, random cropping, color distortion or color adjustment.

4. The task execution method for removing backdoors from a self-distillation model according to claim 1, characterized in that: The process of constructing contrast loss and self-distillation feature map loss to train the first feature extraction model includes: The expression of contrast loss is as follows: Where N is the number of samples in the first unlabeled dataset, q i is the feature vector obtained by the first feature extraction model for the input sample i, is the feature vector obtained by the second feature extraction model as the positive sample of input sample i, is the feature vector obtained by the second feature extraction model as the negative sample of input sample i, τ is the temperature coefficient; The expression of the self-distillation feature map loss is as follows: In the formula, F i is the output feature of the i-th feature layer, F D is the output feature of the deepest feature layer, Represents the sum of the p-th power of the absolute value of the output feature.

5. A task execution method for removing backdoors from a self-distillation model according to claim 1 or 4, characterized in that: The process of updating the second feature extraction model by momentum includes: The expression for momentum update is as follows: i k ←mθ k +(1-m)θ q In the formula, θ k is the parameter of the second feature extraction model, θ q is the parameter of the main encoder, the parameter θ of the main encoder q Updates are back-propagated based on the contrastive loss, where m is the momentum coefficient.

6. The task execution method for removing backdoors from a self-distillation model according to claim 1, characterized in that: The process of constructing contrast loss and self-distillation feature map loss to train the first feature extraction model also includes: Add a fully connected layer after each feature layer of the first feature extraction model so that the output dimension of each feature layer is the same as the output dimension of the first feature extraction model; The process of inputting the amplified labeled data into the trained first feature extraction model also includes: A fully connected network is added after the first feature extraction model to reduce the dimension of the extracted features to the confidence dimension of the classification.

7. The task execution method for removing backdoors from a self-distillation model according to claim 1, characterized in that: Constructing the cross entropy loss to train the first feature extraction model again includes: The expression of cross entropy loss is as follows: Where, L i represents the output of the i-th feature layer, q i represents the confidence output of the i-th feature layer, y represents the final softmax output of the first feature extraction model, C represents the number of categories, j represents the j-th category, j∈C, q ij Represents the confidence output of the i-th feature layer on the j-th category.

8. An electronic device comprising a memory and a processor, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the task execution method for removing the backdoor of the self-distillation model as described in any one of claims 1-7 above.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the task execution method for removing the backdoor of the self-distillation model as described in any one of claims 1-7 is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the task execution method for removing the backdoor of the self-distillation model described in any one of claims 1-7 is implemented.

Images