Deep learning model backdoor attack defense method based on relational knowledge distillation
By using the backdoor model as a teacher model, it performs relational knowledge distillation training, eliminates the backdoor samples, and generates clean data samples, the problem that the deep learning model is affected by the backdoor samples during the training process is solved, effective defense against backdoor attacks is achieved, and the model's security and prediction accuracy are improved.
Patent Information
- Application Number
- CN202411868540.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-18
- Publication Date
- 2025-04-25
AI Technical Summary
Deep learning models are easily affected by backdoor samples during training, resulting in the model embedded in backdoor features, which are difficult to detect and clear, affecting the prediction accuracy and security of the model.
The backdoor attack defense method of deep learning model based on relational knowledge distillation is adopted, and the backdoor model is used as the teacher model. By grouping data samples, the backdoor samples are eliminated, the clean data samples are generated, and the relationship knowledge distillation training is carried out to update the student model parameters to avoid the impact of the backdoor samples.
Effectively prevent deep learning models from being exposed to backdoor samples during training, improve the security and prediction accuracy of the model, reduce the need for accurately removing backdoor samples from the training data set, and improve the universality of defense methods.
Smart Images
Figure CN119885193A_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to the field of machine learning, and in particular to a deep learning model backdoor attack defense method based on relational knowledge distillation. [Background technology]
[0002] Deep machine learning has achieved great success in industrial applications and academic research. With the development of deep machine learning research and application, the security issues of artificial intelligence models have become increasingly prominent, and the vulnerability and robustness of deep learning models have become the focus of attention. Attack methods against deep learning models are endless, among which backdoor attacks are one of the most typical attack methods.
[0003] Adding backdoor triggers to training samples is the most direct and widely used method for attackers to embed backdoors in deep learning models. Attackers modify some training samples by adding specified backdoor triggers, and use the modified samples with the attacker's specified target labels and the remaining benign samples to train the deep learning model, thereby embedding backdoors in the deep learning model. Backdoor attacks are highly concealed. Usually, the existence of backdoor datasets and deep learning models embedded with backdoors cannot be discovered. Deep learning models embedded with backdoors can make correct predictions for benign samples, unlike normal deep learning models. Once the samples input to the model are added with backdoor triggers, the deep learning model embedded with backdoors will predict the samples input to the model as the categories specified by the attacker.
[0004] Existing deep learning technology requires a large number of training samples. The openness of the training data set and the openness of human annotation work allow attackers to easily access the data set used for model training. If a small number of backdoor samples with backdoor triggers that cannot be observed by the naked eye are added to the data set, the purpose of the attack can be achieved. In order to improve the concealment of backdoor attacks, attackers usually only mix a small number of backdoor samples into the data set, making it difficult to completely detect and remove backdoor samples from the data set. Once the data set with backdoor samples is used for model training, the backdoor trigger will embed a shortcut from the trigger to the target category in the model, causing irreversible tampering of the model. If you want to restore the model to a clean model without backdoors, it will affect the model structure and reduce the model prediction accuracy. Therefore, how to prevent the model from being affected by backdoor samples during the training process and effectively restore the model to a clean model without backdoors is of great significance for backdoor attack defense of deep learning models. [Summary of the invention]
[0005] In order to prevent the deep learning model from directly learning backdoor trigger features and backdoor data sets during the training process, which may harm the deep learning model, the present invention provides a deep learning model backdoor attack defense method based on relational knowledge distillation, which uses the backdoor model as a teacher model in the distillation training framework to train a clean student model. The method includes the following steps:
[0006] The backdoor model is used as a teacher model in a knowledge distillation training framework, and a student model corresponding to the teacher model is constructed in the knowledge distillation training framework;
[0007] The training data set samples are grouped to obtain several groups of data samples; each group of data samples is input into the teacher model for processing to obtain relational knowledge of each group of data samples, and backdoor samples are removed to obtain several groups of clean data samples;
[0008] Based on all groups of clean data samples and the relational knowledge, the student model is trained with relational knowledge distillation to determine a first type of loss; based on a data feature graph relationship between the teacher model and the student model, a second type of loss is determined;
[0009] Based on the first type of loss and the second type of loss, model parameters of the student model are updated.
[0010] Preferably, the training data set samples are grouped to obtain several groups of data samples, specifically:
[0011] The N samples included in the training data set samples are randomly divided into N / 2 groups of data samples, wherein each group of data samples includes two samples, and the N samples are an even number of samples obtained from the training data set.
[0012] Preferably, each set of data samples is input into the teacher model for processing to obtain relational knowledge of each set of data samples, specifically:
[0013] Each group of data samples is input into the teacher model to calculate the distance between the sample feature graphs of two samples in each group of data samples; based on the distance between the sample feature graphs of the two samples, the relational knowledge of each group of data samples is obtained.
[0014] Preferably, each group of data samples is input into the teacher model for processing, backdoor samples are removed, and several groups of clean data samples are obtained, specifically:
[0015] Each group of data samples is input into the teacher model to calculate the sample feature graphs of the two samples in each group of data samples; based on the sample feature graphs of the two samples, it is determined whether each group of data samples contains backdoor samples; then all groups of data samples containing backdoor samples are eliminated, and all groups of data samples that do not contain backdoor samples are retained, thereby obtaining several groups of clean data samples.
[0016] Preferably, judging whether each group of data samples contains a backdoor sample based on the sample feature graphs of the two samples is specifically as follows:
[0017] Determining a distance between two sample feature maps based on the respective sample feature maps of the two samples;
[0018] The distance is compared with a preset distance range. If the distance is within the preset distance range, it is determined that each group of data samples does not include a backdoor sample; otherwise, it is determined that each group of data samples includes a backdoor sample.
[0019] Preferably, the student model is trained for relational knowledge distillation based on all groups of clean data samples and the relational knowledge to determine the first type of loss, specifically:
[0020] Based on the group of clean data samples and the relational knowledge, generating relational knowledge data samples, and then using the relational knowledge data samples to perform relational knowledge distillation training on the student model;
[0021] The loss between the output and the data label is predicted by the student model that has completed the relational knowledge distillation training, and this is used as the first type of loss.
[0022] Preferably, the second type of loss is determined based on the data feature graph relationship between the teacher model and the student model, specifically:
[0023] The second type of loss is determined based on the losses corresponding to the distance between the sample feature graphs of two samples in each group of data samples calculated by the teacher model and the distance between the sample feature graphs of two samples in each group of clean data samples calculated by the learning model that completes the relational knowledge distillation training.
[0024] Preferably, the updating of the model parameters of the student model based on the first type of loss and the second type of loss is specifically as follows:
[0025] Based on the first type of loss and the second type of loss, a total loss function of the learning model that completes the relational knowledge distillation training is obtained; and then based on the total loss function, the model parameters of the student model are updated.
[0026] Preferably, the total loss function of the learning model for completing the relational knowledge distillation training is obtained based on the first type of loss and the second type of loss, specifically:
[0027] According to the following formula, based on the first type of loss Loss1 and the second type of loss Loss2, the total loss function Loss of the learning model that completes the relational knowledge distillation training is obtained:
[0028] Loss = Loss1 + λ * Loss2
[0029] In the above formula, λ is the weight coefficient.
[0030] Preferably, the updating of the model parameters of the student model based on the first type of loss and the second type of loss is specifically as follows:
[0031] Based on the total loss function Loss, the model parameters of the student model are updated.
[0032] Compared with the prior art, the present invention has the following beneficial effects:
[0033] The training data set samples are grouped to obtain several groups of data samples; each group of data samples is input into the teacher model for processing to obtain the relational knowledge of each group of data samples, and the backdoor samples are removed to obtain several groups of clean data samples. If the original training data set is used directly to train the teacher model, when the training data set contains backdoor samples, the backdoor trigger will embed a shortcut mapping of the backdoor feature to the target category in the teacher model, thereby embedding the backdoor in the teacher model; the teacher model embedded with the backdoor will show abnormalities when predicting the backdoor samples, for example, the teacher model embedded with the backdoor focuses on the area where the backdoor trigger is located, and the teacher model embedded with the backdoor will output a higher confidence when predicting the backdoor samples. By using the above abnormal behavior presented by the teacher model embedded with the backdoor when predicting the backdoor samples, the backdoor samples of the training data set are screened and removed to prevent the student model from contacting the backdoor samples during the distillation process, thereby defending against backdoor attacks.
[0034] Based on all groups of clean data samples and the relational knowledge, the student model is trained with relational knowledge distillation to determine the first type of loss; based on the data feature graph relationship between the teacher model and the student model, the second type of loss is determined; based on the first type of loss and the second type of loss, the model parameters of the student model are updated. Identify and remove backdoor samples from the training data set samples to obtain several groups of clean data samples, so that the data samples used for subsequent training of the student model are reduced, which will affect the training effect of the student model. Combining all groups of clean data samples and relational knowledge to jointly train the student model can make up for the defect of insufficient number of data samples and improve the training reliability of the student model. By determining the first type of loss for relational knowledge distillation training of the student model and the second type of loss between the teacher model and the student model on the data feature graph, and updating the model parameters of the student model, the accuracy of the student model after backdoor attack defense can be guaranteed.
[0035] In general, the deep learning model backdoor attack defense method based on relational knowledge distillation of the present invention has the following advantages: first, it can prevent the student model from being affected by backdoor samples during the knowledge distillation process; second, there is no need to accurately remove the backdoor samples in the training data set, but to remove the abnormal data sample pairs, so that the backdoor attack defense method is easier to implement and has higher versatility; third, there is no need to restore the model embedded with the backdoor, but to retrain a clean student model without backdoors. Compared with the defense method based on backdoor model recovery in the prior art, it can more effectively ensure the accuracy of the model after backdoor attack defense.
Brief Description of the Drawings
[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:
[0037] Figure 1 It is a flowchart of the deep learning model backdoor attack defense method based on relational knowledge distillation provided by the present invention. [Specific implementation method]
[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention rather than all structures are shown in the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0039] The terms "including" and "having" and any variations thereof in the present invention are intended to cover non-exclusive inclusions. For example, a process, method, method, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.
[0040] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0041] See also Figure 1 As shown, the present invention provides a deep learning model backdoor attack defense method based on relational knowledge distillation, the method comprising the following steps:
[0042] S100, using the backdoor model as a teacher model in a knowledge distillation training framework, and constructing a student model corresponding to the teacher model in the knowledge distillation training framework;
[0043] The training data set samples are grouped to obtain several groups of data samples; each group of data samples is input into the teacher model for processing to obtain the relational knowledge of each group of data samples, and the backdoor samples are eliminated to obtain several groups of clean data samples.
[0044] Furthermore, the training data set samples are grouped to obtain several groups of data samples, specifically:
[0045] The N samples contained in the training data set samples are randomly divided into N / 2 groups of data samples, where each group of data samples includes two samples, and N samples are an even number of samples obtained from the training data set. Among them, N samples are an even number of samples obtained from the training data set, that is, N samples can be all or part of the samples in the training data set, as long as N is an even number. When obtaining an even number of samples from the training data set, each obtained sample can be screened and identified to determine whether the screened sample meets the preset sample content conditions, so as to gather an even number of samples that meet the preset sample content conditions. In addition, the training data set may be, but is not limited to, a training data set corresponding to the deep learning model, and the data type of the samples in the training data set is related to the operating scenario of the deep learning model, such as surveillance image data, temperature and other environmental data in public places, which will not be described in detail here. Teacher model and student model are commonly used model types in knowledge distillation training. The use of teacher model and student model is a conventional technical means in this field, which will not be described in detail here.
[0046] The main purpose of grouping the training data set samples is to determine the relational knowledge of the teacher model. The relational knowledge of the teacher model is obtained by inputting the grouped data samples into the teacher model, and then obtaining the relational knowledge of the teacher model according to the sample feature graph of each group of data samples. In order to better screen out abnormal samples from the training data set and transfer effective relational knowledge to the student model during the knowledge distillation process, the N samples in the training data set can be randomly divided into N / 2 groups of data samples, that is, the N samples are randomly paired in pairs, so that each group of data samples includes two samples, which can be used to find abnormal samples later and calculate relational knowledge during the knowledge distillation process.
[0047] In addition, the more samples there are in each group of data samples, the higher the efficiency of removing backdoor samples. When each group of data samples contains more samples, each sample will be compared with more other samples in the process of calculating the distance between the corresponding sample feature graphs, so it is more conducive to the discovery and removal of abnormal samples. At the same time, the more samples there are in each group of data samples, the greater the difference between the performance of the trained student model and the performance of the teacher model. In order to ensure the removal effect of backdoor samples, when it is found that the corresponding group of data samples contains abnormal data, all the corresponding group of data samples will be removed. Therefore, when the number of samples contained in each group of data samples is larger, the more benign samples will be removed while removing the corresponding group of data samples containing abnormal data, thereby affecting the amount of training data for training the student model. Through the above analysis, including two samples in each group of data samples can simultaneously improve the removal efficiency of backdoor samples and avoid excessive removal of benign samples.
[0048] Furthermore, each set of data samples is input into the teacher model for processing to obtain the relational knowledge of each set of data samples, specifically:
[0049] Each group of data samples is input into the teacher model to calculate the distance between the sample feature graphs of two samples in each group of data samples; based on the distance between the sample feature graphs of two samples, the relational knowledge of each group of data samples is obtained.
[0050] Each set of data samples is input into the teacher model to calculate the distance between the sample feature graphs f1 and f2 of the two samples in each set of data samples. The above process can be expressed by the following formula:
[0051] D(f1,f2)=||f1,f2|| 2
[0052] In the above formula, D(f1, f2) is the distance between the sample feature graphs f1 and f2 of the two samples in each group of data samples, |||| 2 It is the distance operation between two sample feature maps.
[0053] Then, based on the distance between the sample feature graphs of the two samples, the knowledge relationship between the two samples under each group of data samples is constructed to obtain the relational knowledge of each group of data samples.
[0054] Furthermore, each set of data samples is input into the teacher model for processing, and the backdoor samples are removed to obtain several sets of clean data samples, specifically:
[0055] Each group of data samples is input into the teacher model to calculate the sample feature graphs of the two samples in each group of data samples; based on the sample feature graphs of the two samples, it is determined whether each group of data samples contains backdoor samples; then all group data samples containing backdoor samples are eliminated, and all group data samples that do not contain backdoor samples are retained, thereby obtaining several groups of clean data samples.
[0056] Furthermore, based on the sample feature graphs of the two samples, it is determined whether each group of data samples contains a backdoor sample, specifically:
[0057] Determine a distance between two sample feature maps based on the respective sample feature maps of the two samples;
[0058] The distance is compared with a preset distance range. If the distance is within the preset distance range, it is determined that each group of data samples does not contain a backdoor sample; otherwise, it is determined that each group of data samples contains a backdoor sample.
[0059] If one of the two samples contained in the corresponding group of data samples is a backdoor sample and the other is a benign sample, due to the abnormal attention shown by the teacher model to the backdoor sample, there is a huge difference between the sample feature map corresponding to the backdoor sample and the sample feature map corresponding to the benign sample. At this time, the distance between the two sample feature maps calculated is large, so the corresponding group of data samples will be completely removed; if the two samples contained in the corresponding group of data samples are both backdoor samples, since the model shows the same degree of abnormal attention to the same backdoor trigger, the difference between the sample feature maps corresponding to the two backdoor samples is small, and the distance between the two sample feature maps calculated is small, so the corresponding group of data samples will be completely removed; if the two samples contained in the corresponding group of data samples are both benign samples, since the model will show different attention to different features in different benign samples, there is no abnormality in the distance between the two sample feature maps calculated at this time, so the corresponding group of data samples will be retained to form clean data samples.
[0060] In actual operation, a preset distance range is set, wherein the preset distance range includes an upper distance value and a lower distance value; when the distance between two sample feature maps is not within the preset distance range, for example, the distance between the two sample feature maps is greater than the upper distance value of the preset distance range, it indicates that one of the two samples contained in the corresponding group of sample data is a backdoor sample and the other is a benign sample, so the corresponding group of data samples needs to be completely removed; for example, the distance between the two sample feature maps is less than the lower distance value of the preset distance range, indicating that the two samples contained in the corresponding group of sample data are both backdoor samples, so the corresponding group of data samples need to be completely removed. When the distance between the two sample feature maps is within the preset distance range, it indicates that the two samples contained in the corresponding group of sample data are both benign samples, so the corresponding group of data samples need to be retained.
[0061] In general, by utilizing the abnormal behavior of the teacher model embedded with a backdoor when predicting backdoor samples, the backdoor samples in the training data set are screened out to prevent the student model from being exposed to backdoor samples during the distillation process, thereby defending against backdoor attacks.
[0062] S200, based on all groups of clean data samples and relational knowledge, the student model is trained for relational knowledge distillation to determine the first type of loss; based on the data feature graph relationship between the teacher model and the student model, the second type of loss is determined.
[0063] Furthermore, based on all groups of clean data samples and relational knowledge, the student model is trained for relational knowledge distillation to determine the first type of loss, which is:
[0064] Based on the group of clean data samples and relational knowledge, relational knowledge data samples are generated, and then the relational knowledge data samples are used to perform relational knowledge distillation training on the student model;
[0065] The loss between the output and the data label predicted by the student model that has completed relational knowledge distillation training is used as the first type of loss.
[0066] Furthermore, based on the data feature graph relationship between the teacher model and the student model, the second type of loss is determined, specifically:
[0067] The second type of loss is determined based on the losses corresponding to the distance between the sample feature maps of two samples in each group of data samples calculated by the teacher model and the distance between the sample feature maps of two samples in each group of clean data samples calculated by the learning model that completes the relational knowledge distillation training.
[0068] Furthermore, based on the first type of loss and the second type of loss, the model parameters of the student model are updated, specifically:
[0069] Based on the first and second type losses, the total loss function of the learning model that completes the relational knowledge distillation training is obtained; then based on the total loss function, the model parameters of the student model are updated.
[0070] Furthermore, based on the first type of loss and the second type of loss, the total loss function of the learning model that completes the relational knowledge distillation training is obtained, specifically:
[0071] According to the following formula, based on the first type of loss Loss1 and the second type of loss Loss2, the total loss function Loss of the learning model that completes the relational knowledge distillation training is obtained:
[0072] Loss = Loss1 + λ * Loss2
[0073] In the above formula, λ is the weight coefficient.
[0074] Identify and remove backdoor samples from the training data set samples to obtain several groups of clean data samples, so that the data samples used for subsequent training of the student model become fewer, which will affect the training effect of the student model. Combining all groups of clean data samples and relational knowledge to jointly train the student model can make up for the defect of insufficient number of data samples and improve the training reliability of the student model.
[0075] For a data training set with backdoor samples added, the data features it contains are divided into backdoor features and benign features. Among them, the backdoor features and their data labels are often single and unchanged, that is, the image of the backdoor trigger, whose label is set to the category that the attacker wants to attack, while the benign features and their data labels are changeable, and the model needs to find the features of each category of targets and the complex mapping relationship between the category data labels. Therefore, compared with benign features, it is easier for the model to establish a strong mapping relationship between backdoor features and their data labels. When the model predicts backdoor samples, it will show attention anomalies in the sample feature graph. Through relational knowledge distillation, the distance between the sample feature graphs of each group of data samples is compared to screen out the corresponding group of data samples containing abnormal data, thereby avoiding the corresponding group of data samples containing abnormal data in the knowledge distillation process from being input into the student model, and achieving effective defense against backdoor attacks.
[0076] After training the student model using relational exponential distillation, the student model predicts the output y pr With data label y gt The loss Loss1 between the two samples is used as the first type of loss. The distance D between the sample feature maps of each group of data samples calculated based on the teacher model is also T And the distance D between the sample feature graphs of two samples in each group of clean data samples calculated by the learning model that completes the relational knowledge distillation training S The corresponding loss Loss2 is used as the second type of loss. Specifically, the total loss function Loss of the learning model that completes the relational knowledge distillation training can be calculated by the following formula:
[0077] Loss=Loss1+λ*Loss2=∑ M (y pr -y gt ) 2 +λ*∑ M (D T -D S ) 2
[0078] In the above formula, λ is the weight coefficient, and M is the number of clean data sample groups.
[0079] S300, based on the first type of loss and the second type of loss, update the model parameters of the student model.
[0080] Furthermore, based on the first type of loss and the second type of loss, the model parameters of the student model are updated, specifically:
[0081] Based on the total loss function Loss, the model parameters of the student model are updated.
[0082] By updating the model parameters of the student model through the total loss function Loss, the parameters of the student model can be adjusted and improved more accurately, so that the student model can be improved in the direction of effectively reducing the loss, further improving the performance of the student model.
[0083] Through the description of the above implementation methods, technicians in this field can clearly understand that each implementation method can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it, and other embodiments may also be used. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A deep learning model backdoor attack defense method based on relational knowledge distillation, characterized in that: The method comprises the following steps: The backdoor model is used as a teacher model in a knowledge distillation training framework, and a student model corresponding to the teacher model is constructed in the knowledge distillation training framework; The training data set samples are grouped to obtain several groups of data samples; each group of data samples is input into the teacher model for processing to obtain relational knowledge of each group of data samples, and backdoor samples are removed to obtain several groups of clean data samples; Based on all groups of clean data samples and the relational knowledge, the student model is trained with relational knowledge distillation to determine a first type of loss; based on a data feature graph relationship between the teacher model and the student model, a second type of loss is determined; Based on the first type of loss and the second type of loss, model parameters of the student model are updated.
2. The method according to claim 1, characterized in that The training data set samples are grouped to obtain several groups of data samples, specifically: The N samples included in the training data set samples are randomly divided into N / 2 groups of data samples, wherein each group of data samples includes two samples, and the N samples are an even number of samples obtained from the training data set.
3. The method according to claim 2, characterized in that Each group of data samples is input into the teacher model for processing to obtain the relational knowledge of each group of data samples, specifically: Each group of data samples is input into the teacher model to calculate the distance between the sample feature graphs of two samples in each group of data samples; based on the distance between the sample feature graphs of the two samples, the relational knowledge of each group of data samples is obtained.
4. The method according to claim 3, characterized in that Each group of data samples is input into the teacher model for processing, and the backdoor samples are removed to obtain several groups of clean data samples, specifically: Each group of data samples is input into the teacher model to calculate the sample feature graphs of the two samples in each group of data samples; based on the sample feature graphs of the two samples, it is determined whether each group of data samples contains backdoor samples; then all groups of data samples containing backdoor samples are eliminated, and all groups of data samples that do not contain backdoor samples are retained, thereby obtaining several groups of clean data samples.
5. The method according to claim 4, characterized in that The determining whether each group of data samples contains a backdoor sample based on the sample feature graphs of the two samples is specifically as follows: Determining a distance between two sample feature maps based on the respective sample feature maps of the two samples; The distance is compared with a preset distance range. If the distance is within the preset distance range, it is determined that each group of data samples does not include a backdoor sample; otherwise, it is determined that each group of data samples includes a backdoor sample.
6. The method according to claim 1, characterized in that Based on all the groups of clean data samples and the relational knowledge, the student model is trained for relational knowledge distillation to determine the first type of loss, which is specifically: Based on the group of clean data samples and the relational knowledge, generating relational knowledge data samples, and then using the relational knowledge data samples to perform relational knowledge distillation training on the student model; The loss between the output and the data label is predicted by the student model that has completed the relational knowledge distillation training, and this is used as the first type of loss.
7. The method according to claim 6, characterized in that The second type of loss is determined based on the data feature graph relationship between the teacher model and the student model, specifically: The second type of loss is determined based on the losses corresponding to the distance between the sample feature graphs of two samples in each group of data samples calculated by the teacher model and the distance between the sample feature graphs of two samples in each group of clean data samples calculated by the learning model that completes the relational knowledge distillation training.
8. The method according to claim 1, characterized in that The updating of the model parameters of the student model based on the first type of loss and the second type of loss is specifically as follows: Based on the first type of loss and the second type of loss, a total loss function of the learning model that completes the relational knowledge distillation training is obtained; and then based on the total loss function, the model parameters of the student model are updated.
9. The method according to claim 8, characterized in that The total loss function of the learning model that completes the relational knowledge distillation training is obtained based on the first type of loss and the second type of loss, specifically: According to the following formula, based on the first type of loss Loss1 and the second type of loss Loss2, the total loss function Loss of the learning model that completes the relational knowledge distillation training is obtained: Loss = Loss1 + λ * Loss2 In the above formula, λ is the weight coefficient.
10. The method according to claim 9, characterized in that The updating of the model parameters of the student model based on the first type of loss and the second type of loss is specifically as follows: Based on the total loss function Loss, the model parameters of the student model are updated.