Model backdoor defense method and device, electronic equipment, storage medium and program

Abstract

The invention discloses a model backdoor defense method and device, electronic equipment, a storage medium and a program, and the method comprises the steps: obtaining a pure data set of a target model, and carrying out the backdoor erasing of the target model based on the pure data set, and enabling the target model to serve as a teacher model; performing data enhancement on the pure data set as a distillation training data set; and performing distillation training on the target model according to the distillation training data set and the teacher model to obtain a safety model. According to the embodiment of the invention, the backdoor of the model can be cleared, the universality of the backdoor clearing process is improved, the prior knowledge of the backdoor model does not need to be learned, and the use safety of the model can be improved.

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and in particular to a model backdoor defense method, device, electronic device, storage medium and program. Background Art

[0002] With the rise of the big data era and the continuous breakthroughs in artificial intelligence (AI) theory and technology, deep learning, as a core technology in this field, has been widely applied in fields such as computer vision, natural language processing, and speech recognition, achieving decisive success. Natural language processing (NLP) models are a key component of deep learning data. They have made significant progress in tasks such as machine translation and sentiment analysis, and have spawned a series of practical applications such as malicious text detection and spam filtering. Research has shown that NLP models are extremely vulnerable to backdoor attacks. Attackers can publish training sets containing poisoned samples online, misleading users into using the poisoned samples to train backdoor models and deploy them on their own application systems. Because the output of a backdoor model for clean samples is the same as that of the clean model, and the poisoned samples are constructed in a diverse and hidden manner, it is very difficult to completely defend against backdoor attacks.

[0003] Numerous studies have been conducted on backdoor removal methods for NLP models, but these methods are not fully suitable for defending against various backdoor models. This is due to the following three reasons: 1. Existing NLP model backdoor attacks are diverse and subtle, making current backdoor defense methods easily circumvented by new types of backdoor attacks; 2. Existing backdoor removal methods often require large amounts of clean data to repeatedly train the backdoor models, which is often unavailable to ordinary users; and 3. Backdoor removal methods are highly model-dependent, requiring users to understand the model prior to purifying existing backdoor models. A model backdoor defense method is urgently needed to address these issues. Summary of the Invention

[0004] The present invention provides a model backdoor defense method, device, electronic device, storage medium and program to clear the model backdoor, improve the versatility of the clearing process, eliminate the need to learn prior knowledge of the backdoor, and improve the security of model use.

[0005] According to one aspect of the present invention, a model backdoor defense method is provided, wherein the method comprises:

[0006] Obtaining a clean data set of a target model, and performing backdoor erasure on the target model based on the clean data set as a teacher model;

[0007] Performing data augmentation on the clean dataset as a distillation training dataset;

[0008] The target model is distilled and trained according to the distilled training data set and the teacher model to obtain a security model.

[0009] According to another aspect of the present invention, a model backdoor defense device is provided, wherein the device includes:

[0010] A teacher acquisition module is used to obtain a clean data set of a target model, and perform backdoor erasure on the target model based on the clean data set as a teacher model;

[0011] A data preparation module, configured to perform data augmentation on the clean dataset as a distillation training dataset;

[0012] The model distillation module is used to perform distillation training on the target model based on the distillation training dataset and the teacher model to obtain a security model.

[0013] According to another aspect of the present invention, an electronic device is provided, comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the model backdoor defense method described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the model backdoor defense method described in any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present application obtains a pure data set of the target model, erases the backdoor of the target model according to the pure data, and uses the erased target model as a teacher model to perform data enhancement on the pure data to obtain a distillation training data set, and performs distillation training on the target model according to the distillation training data and the teacher model to obtain a security model. The embodiment of the present application erases the backdoor of the pure data set of the target model, so that the corresponding teacher model is generated by the model itself. No additional prior knowledge is required to generate the teacher model, which reduces the difficulty of generating the teacher model. The pure data set is enhanced into a distillation training data set, which can meet the data demand of the backdoor defense process and ensure the effect of clearing the model backdoor.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 This is a flow chart of a model backdoor defense method provided according to the first embodiment of the present application;

[0022] Figure 2 This is a flowchart of another model backdoor defense method provided in accordance with the second embodiment of the present application;

[0023] Figure 3 This is a flowchart of another model backdoor defense method provided in accordance with the third embodiment of the present application;

[0024] Figure 4 This is an example diagram of a model backdoor defense method provided according to the fourth embodiment of the present application;

[0025] Figure 5 This is a schematic structural diagram of a model backdoor defense device provided according to the fifth embodiment of the present application;

[0026] Figure 6 It is a structural diagram of an electronic device that implements the model backdoor defense method of an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Example 1

[0030] Figure 1 This is a flow chart of a model backdoor defense method provided in accordance with the first embodiment of the present application. This embodiment is applicable to the case of model backdoor removal. The method can be executed by a model backdoor defense device, which can be implemented in the form of hardware and / or software and can be configured in a server or server cluster. Figure 1 As shown, the method includes:

[0031] Step 110: Obtain a clean data set of the target model, and erase the backdoor of the target model based on the clean data set to serve as a teacher model.

[0032] Among them, the clean dataset can be a data set used to backdoor-erase the target model. The clean dataset can be business data collected locally for the target model, or training data provided by a trusted third party. Backdoor erasure can be a defense measure against backdoor attacks on the target model. Backdoor erasure can be implemented through parameter fine-tuning, parameter initialization, neuron pruning, etc. The teacher model can be a model used for distillation training of the target model. The teacher model can provide the target model with behaviors to be imitated. The teacher model can be generated by backdoor-erasing the target model. The number of teacher models can be one or more, and different teacher models can be obtained by backdoor-erasing the target model using different backdoor-erasing methods.

[0033] In an embodiment of the present application, for a target model to be processed, wherein the target model may be a neural network model to be backdoor-erased, the specific model architecture of the target model may not be limited, and the target model may include but is not limited to a natural language processing (NLP) model, a recurrent neural network model, a graph neural network model, a convolutional neural network model, an attention mechanism model, etc., a corresponding clean data set may be obtained, and the clean data set may be used to backdoor-erased the target model, and the data in the clean data set is secure data, which may be business data collected locally, or training data provided by a trusted third party. The target model may be backdoor-erased according to the obtained clean data set, and the backdoor-erasing method may be one or more, and each target model after backdoor-erasing may be used as a teacher model. The above-mentioned backdoor erasure may include fine-tuning the target model using a clean data set, pruning the neurons of the target model using a clean data set, and initializing the model parameters of the target model using a clean data set.

[0034] Step 120: Perform data enhancement on the clean dataset to obtain a distillation training dataset.

[0035] Among them, data enhancement can be a processing method that increases the amount of data by various transformations or processing of data in a pure data set. Data enhancement can be achieved through synonym replacement, sentence reorganization, random insertion, random deletion, text generation, translation, etc.

[0036] In an embodiment of the present application, data enhancement can be performed on a pure data set. Data enhancement can be performed by performing synonym replacement, sentence reorganization, random insertion, random deletion, text generation, translation, etc. on the data in the pure data set, thereby increasing the data size of the pure data set. The pure data set after data enhancement can be used as a distillation training data set, and the distillation training data set can be used for distillation training of the target model.

[0037] Step 130: Perform distillation training on the target model based on the distillation training dataset and the teacher model to obtain a security model.

[0038] Among them, distillation training can be to use the output of the teacher model as a supervision signal to train the target model. The target model imitates the output probability distribution of the teacher model. The target model adjusts its own parameters during the distillation training process to approach the performance of the teacher model. Distillation training can transfer the knowledge of the teacher model to the target model.

[0039] In an embodiment of the present application, the distillation training data set can be input into the teacher model and the target model respectively, the output result of the teacher model can be obtained, and the model parameters of the target model can be adjusted according to the output result, so that the probability distribution of the output result of the target model is the same as or similar to the probability distribution of the output result of the teacher model, and the target model after distillation training can be used as a security model.

[0040] In an embodiment of the present application, a pure data set of a target model is acquired, the backdoor of the target model is wiped according to the pure data, and the erased target model is used as a teacher model to perform data enhancement on the pure data to obtain a distilled training data set. The target model is distilled and trained according to the distilled training data and the teacher model to obtain a security model. In an embodiment of the present application, the backdoor is erased by the pure data set of the target model, so that the corresponding teacher model is generated by the model itself. No additional prior knowledge is required to generate the teacher model, which reduces the difficulty of generating the teacher model. The pure data set is enhanced into a distilled training data set, which can meet the data requirements of the backdoor defense process and ensure the effect of clearing the model backdoor.

[0041] Example 2

[0042] Figure 2 This is a flowchart of another model backdoor defense method provided in Example 2 of this application. This embodiment of the application is a specific implementation based on the above embodiment. Figure 2 The method provided in the embodiment of the present application specifically includes the following steps:

[0043] Step 210: Obtain an original benign training data set of the target model, and extract a clean data set from the original benign training data set, wherein the data distribution of the clean data set is consistent with the data distribution of the original benign training data set.

[0044] The original benign training data set may be a data set originally used to train the target model, and the original benign training data set may be provided by a trusted third party.

[0045] In an embodiment of the present application, the original benign training data set of the target model can be obtained, and a pure data set can be extracted from the original benign training data set. The data distribution of the pure data set can be consistent with the data distribution of the original benign data set. Specifically, the method of extracting the pure data set from the original benign training data set can be implemented by random stratified sampling, density-based sampling, generative adversarial network or diffusion model, etc. For example, taking random stratified sampling as an example, the proportion of samples of each type in the original benign training data set can be determined, and samples can be randomly extracted from each category in the same proportion to form a pure data set. For another example, taking density-based sampling as an example, K-Means clustering can be performed on the original benign training data set. Each cluster in the clustering result can represent a local pattern of the data in the original benign training data set. Samples can be randomly extracted from each cluster according to a preset proportion to form a pure data set.

[0046] Step 220: Erasing the backdoor of the target model based on the clean data set to obtain a teacher model.

[0047] In an embodiment of the present application, the target model can be backdoor-erased according to a clean data set, and the target model after backdoor erasure can be used as a teacher model. It is understandable that there can be multiple methods for backdoor erasing the target model. After the target model is processed by a backdoor erasing method, it can be used as a teacher model respectively. That is, there can be at least two teacher models in the embodiment of the present application. The specific implementation method of backdoor erasing the target model is not limited here, such as model fine-tuning, parameter initialization, and model pruning.

[0048] Step 230: Obtain the constituent words of each sentence in the clean data set.

[0049] A sentence can be a natural language sentence that constitutes a clean dataset. This sentence can be used in daily human communication, conform to grammatical and semantic rules, and can be used to express a complete meaning. A constituent word can be a constituent element of a sentence. A constituent word can be the smallest unit in a sentence with independent meaning and grammatical function. Sentences can be generated into constituent words through word segmentation.

[0050] In an embodiment of the present application, each sentence included in the pure data set can be extracted, and each sentence can be segmented separately to obtain the constituent words of each sentence in the pure data set. Exemplarily, the word segmentation processing can be implemented through a rule-based word segmentation method, such as a forward maximum matching method, a reverse maximum matching method, a two-way maximum matching method, etc. The word segmentation processing can also be implemented through a statistical-based word segmentation method, such as a hidden Markov model, a conditional random field-based, an N-gram language model, etc. The word segmentation processing can be implemented based on deep learning.

[0051] Step 240: Perform data enhancement on the sentences in the clean dataset based on the synonyms of the constituent words to obtain a distilled training dataset.

[0052] Among them, synonyms can be words with similar or close semantics in each constituent word, and replacing each other in a specific context will not significantly change the core meaning of the sentence.

[0053] In an embodiment of the present application, synonyms for constituent words can be obtained. These synonyms can be determined from a thesaurus with a professional domain background. Data enhancement can be performed on sentences in a clean dataset using the synonyms of each constituent word, thereby increasing text diversity while maintaining the original meaning of the sentences in the clean dataset. For example, synonyms can be used to replace corresponding constituent words in sentences in the clean dataset to form new sentences that retain the semantics of the original sentences.

[0054] Step 250: Perform distillation training on the target model based on the distillation training dataset and the teacher model to obtain a security model.

[0055] In an embodiment of the present application, a clean data set is extracted from the original benign training data set of the target model, and the data distribution of the clean data set is consistent with the data distribution of the original benign training data set. The target model is backdoor-erased according to the clean data set, and the target model after backdoor erasure is used as a teacher model. The constituent words of the sentences are obtained in the clean data set, and the synonyms of the constituent words are determined. The sentences in the clean data set are data-enhanced according to the synonyms. The clean data set after data enhancement is used as a distillation training data set, and the target model is distilled and trained according to the teacher model and the distillation data set, so that the trained target model is used as a security model. The embodiments of the present application extract a pure data set from the original benign data set, thereby ensuring that the data distribution of the pure data set is consistent with the original training data distribution of the target model, ensuring that the original function of the target model is not affected after the backdoor is removed, and improving the stability of the target model; performing data enhancement on the pure data set based on synonyms, the data volume of the pure data set can be increased while ensuring the original semantics of the sentences in the data set, thereby ensuring the data volume of clean data during the model backdoor erasure process and ensuring the backdoor erasure effect; by clearing the model backdoor, the versatility of the clearing process is improved, and there is no need to learn prior knowledge of the backdoor, which can improve the security of model use.

[0056] Based on the above embodiments of the invention, the target model is backdoor-erased according to the clean data set to obtain a teacher model, including at least one of the following:

[0057] Retrain the target model based on the clean dataset and use the retrained target model as the teacher model. This solution can be referred to as fine-tuning.

[0058] Initialize at least some of the model weight parameters in the target model, retrain the initialized target model based on clean data, and use the retrained target model as the teacher model, i.e., parameter initialization;

[0059] The target model is tested according to the clean data set, the target neurons in the target model whose output information is less than the activation threshold are determined, the target neurons in the target model are removed, and the target model is retrained as the teacher model according to the clean data set, which is also called pruning.

[0060] In an embodiment of the present application, the method of backdoor erasing the target model may include retraining the target model, initializing the model weight parameters of the target model, and pruning the neurons of the target model.

[0061] Specifically, retraining the target model may include retraining the target model on a clean dataset, and the retrained target model may be used as a teacher model. This backdoor erasing process may be called fine-tuning.

[0062] The backdoor erasing method can also include initializing the model weight parameters of the target model. Since the toxic weights of the model are mainly concentrated in the high-level model, some or all of the model weight parameters can be selected in the target model as the model weight parameters to be adjusted through configuration or indication. The model weight parameters to be adjusted selected in the target model can be initialized. The initialization can be achieved by random assignment or assignment according to a configuration file. For the target model after the model weight parameters are initialized, the target model can be retrained using a pure data set, and the retrained target model can be used as a teacher model. The backdoor erasing process can be called parameter initialization.

[0063] Pruning the neurons of the target model can include inputting a clean data set into the target model, monitoring the output information of each neuron in the target model, and judging whether the output information of the neurons in the target model is less than an activation threshold. The activation threshold can be a pre-configured output critical value. When the output information of the neuron is greater than or equal to the activation threshold, the neuron of the target model can be determined to be in an activated state. The output information of each neuron can be compared with the activation threshold to determine the target neuron in the target model. The target neuron in the target model can be removed, and the target model after removing the target neuron is trained using the clean data set to obtain a teacher model. This backdoor erasing process can be called pruning.

[0064] Based on the above application embodiment, data enhancement is performed on sentences in the clean dataset based on synonyms of the constituent words, including:

[0065] Determine the importance parameter value of each constituent word, and select at least some of the constituent words according to the importance parameter value to construct an important word list; determine the target words to be replaced that do not belong to the important word list in each sentence in the clean data set; generate synonyms of the target words to be replaced, and replace the target words to be replaced in each sentence according to the synonyms.

[0066] The importance parameter value may be a parameter reflecting the importance of a constituent word within the sentence to which it belongs. The importance parameter value may be achieved by performing word frequency-inverse document frequency analysis, word embedding feature analysis, syntactic analysis, TextRank algorithm analysis, and the like on the sentence. The important word list may be the names of constituent words selected from a clean dataset using the importance parameter value. The importance parameter value of the constituent words in the reused word list may be greater than or equal to a threshold value. That is, the constituent words included in the important word list may be relatively important words, for example, the constituent words may be words that reflect the semantics of the sentence.

[0067] In an embodiment of the present application, the importance parameter of each constituent word can be determined by filtering the sentences in the clean data set through part-of-speech, statistical feature analysis, or semantic and context analysis. The importance parameter can be determined by at least one of the indicators such as word frequency, word frequency-inverse document frequency, or part-of-speech. The constituent words can be filtered according to the importance parameter, and constituent words with importance parameters greater than a threshold can be selected to form an important word list. Each sentence in the clean data set can be filtered based on the important word list, and the constituent words that do not belong to the important word list can be determined in the sentence as the target word to be replaced. It can be understood that in each sentence, at least one constituent word that does not belong to the important word list can be selected as the target word to be replaced, and synonyms of the target word to be replaced can be generated. The synonyms can be generated through a dictionary or a domain-related vocabulary database. The target word to be replaced in the sentence can be replaced with the determined synonym to form a new sentence, thereby achieving enhancement of the clean data set.

[0068] Based on the above application embodiment, data enhancement is performed on sentences in the clean dataset based on synonyms of the constituent words, including:

[0069] Randomly select constituent words in the clean dataset as target words; generate synonyms of the target words, and randomly insert the synonyms into the sentences corresponding to the target words in the clean dataset.

[0070] In an embodiment of the present application, constituent words can be selected from each sentence in a clean data set as target vocabulary, synonyms of the target vocabulary can be generated through a dictionary or a domain-related vocabulary database, and synonyms can be randomly inserted into sentences to which the target vocabulary belongs in the clean data set to form new sentences, thereby enhancing the clean data set.

[0071] Example 3

[0072] Figure 3 This is a flowchart of another model backdoor defense method provided in Example 3 of this application. This embodiment of the application is a concretization based on the above embodiment, and describes the distillation training process of the target model. Figure 3 The method provided in the embodiment of the present application specifically includes the following steps:

[0073] Step 310: Obtain a clean data set of the target model, and erase the backdoor of the target model based on the clean data set as a teacher model.

[0074] Step 320: Perform data enhancement on the clean dataset to serve as a distillation training dataset.

[0075] Step 330: Input the distillation training dataset into the target model and the teacher model respectively.

[0076] In an embodiment of the present application, the training data in the distillation training data set can be input into the target model and the teacher model respectively, so that the target model and the teacher model process the input training data respectively.

[0077] Step 340: Obtain the first model layer output result of the intermediate layer of the target model and the second model layer output result of the intermediate layer of the teacher model, and obtain the first model output result of the target model and the second model output result of the teacher model.

[0078] The intermediate layer can be the core part of the target model that processes input data and extracts high-level features. The intermediate layer has functions such as local feature abstraction, information transfer, and context modeling. Taking the target model as an NLP model as an example, the intermediate layer can be the time-step recurrent unit of the NLP model. Taking the target model as an attention mechanism model as an example, the intermediate layer can be the attention layer. It is understood that the examples here are for illustrative purposes only and are not limiting. In other types of neural network models, the intermediate layer can be a structure corresponding to the neural network model type.

[0079] In an embodiment of the present application, the output results of the intermediate layer of the target model and the intermediate layer of the teacher model for the training data in the input distillation training data set can be collected respectively, the output result of the intermediate layer of the target model can be recorded as the first model layer output result, the output result of the intermediate layer of the teacher model can be recorded as the second model layer output result, and the model output results of the target model and the teacher model can also be collected and recorded as the first model output result and the second model output result respectively.

[0080] Step 350: Determine a preset loss value based on the output result of the first model layer, the output result of the second model layer, the output result of the first model, and the output result of the second model.

[0081] In an embodiment of the present application, the information of the intermediate layer of the teacher model and the final output of the model can be used as target information, and the loss values between the first model layer output result of the intermediate layer of the target model and the first model output result of the model output and the second model layer output result of the intermediate layer of the teacher model and the second model output result of the teacher model respectively can be determined, and the loss value corresponding to the intermediate layer and the final output result of the model can be determined as a preset loss value, wherein the method for determining the preset loss value based on the loss value of the intermediate layer and the loss value of the final output result of the model can include determining it by summing and averaging. It can be understood that the loss value between the target model and the intermediate layer and the final output result of the teacher model can be determined by a loss function, and the loss function can include but is not limited to a cross entropy loss function, a contrast loss function, a mean square error loss function, a cosine similarity loss function, a log-likelihood loss function, and the like.

[0082] Step 360: When the preset loss value meets the specified conditions, it is determined that the target model training is completed, and the target model is used as a security model.

[0083] Among them, the preset specified conditions can be the conditions for judging whether the training of the target model is completed, and the preset specified conditions can include the maximum number of training iterations of the target model, the loss value threshold, etc.

[0084] In an embodiment of the present application, specified conditions can be obtained, and the number of training iterations corresponding to the preset loss value and whether the preset loss value meets the maximum number of training iterations and loss value threshold of the specified conditions can be determined. If it is determined that the preset loss value meets the specified conditions, the target model training can be completed and the target model can be used as a security model.

[0085] In an embodiment of the present application, a pure data set of a target model is obtained, backdoors of the target model are erased according to the pure data set as a teacher model, the pure data set is used as a distillation training data set after data enhancement, and the distillation training data set is input into the target model and the teacher model respectively, and the first model layer output results and the second model layer output results of the corresponding intermediate layers of the target model and the teacher model are obtained, as well as the first model output results and the second model output results of the model's final output, and a preset loss value is determined based on the first model layer output results, the second model layer output results, the first model output results and the second model output results, and when the preset loss value meets the preset conditions, the target model is used as a security model. In an embodiment of the present application, backdoors are erased through the pure data set of the target model, so that the corresponding teacher model is generated by the model itself, and no additional prior knowledge is required to generate the teacher model, thereby reducing the difficulty of generating the teacher model, and enhancing the pure data set into a distillation training data set, which can meet the data requirements of the backdoor defense process and ensure the effect of clearing the model backdoor.

[0086] Furthermore, based on the above-mentioned application embodiment, the preset loss value is determined according to the output result of the first model layer, the output result of the second model layer, the output result of the first model, and the output result of the second model, including:

[0087] Step 3501: Based on the preset activation function, the output result of the first model is processed into the sample prediction probability of each sample in the corresponding distillation training data set, the true label information of each sample in the distillation training data set is determined, and the first cross entropy loss value of the target model is determined according to the true label information and sample prediction probability of each sample.

[0088] The preset activation function may be a function that processes the model output result into a prediction probability, and the preset activation function may include a Sigmoid function, a Softmax function, etc. The sample prediction probability may output the prediction probability that the result belongs to a specific sample, and the true label may be information reflecting that the sample belongs to a specific classification. The true label may be a sample label in the distillation training dataset.

[0089] In an embodiment of the present application, the output result of the first model can be processed by a preset activation function, the output result of the first model can be converted into a sample prediction probability of the corresponding sample, the true label information of each sample in the distillation training data set can be extracted, and the cross entropy loss value can be calculated by the true label information of each sample and the sample prediction probability. The cross entropy loss value can be used as the first cross entropy loss value of the target model. Specifically, the calculation formula of the cross entropy loss value can be as follows:

[0090]

[0091] Among them, L CErepresents the cross entropy loss value, N represents the number of samples in the distillation training data set, i is an ordinal number ranging from 1 to N, and y i represents the true label of the i-th sample input to the target model, Represents the sample prediction probability of the target model s for the i-th sample.

[0092] Step 3502: Obtain the preset distillation temperature parameters, and process the first model output results and the second model output results into the first distilled sample prediction probability and the second distilled sample prediction probability of the corresponding samples based on the preset activation function and the preset distillation temperature parameters, and determine the second cross entropy loss value for the teacher model based on the first distilled sample prediction probability and the second distilled sample prediction probability of each sample.

[0093] Among them, the preset distillation temperature parameter can be a pre-configured parameter value, which can adjust the output probability distribution between the target model and the teacher model. When the value of the preset distillation temperature parameter increases, the output probability distribution can become smoother, that is, the probability difference of each classification is reduced.

[0094] In an embodiment of the present application, the first model output result of the target model and the second model output result of the teacher model can be processed separately by presetting the distillation temperature parameter, and the first model output result and the second model output result can be adjusted separately. The adjustment method can include determining the quotient of the first model output result and the second model output result with the preset distillation temperature parameter, or weighting, etc. The adjusted first model output result and the second model output result can be input into the preset activation function to obtain the corresponding first distillation sample prediction probability and the second distillation sample prediction probability, and the cross entropy loss value is calculated according to the first distillation sample prediction probability and the second distillation sample prediction probability, so as to obtain the second cross entropy loss value of the teacher model. Specifically, the process of calculating the cross entropy loss value according to the first distillation sample prediction probability and the second distillation sample prediction probability can be achieved by the following formula:

[0095]

[0096] L KD It can represent the cross entropy loss value, T represents the preset distillation temperature parameter, represents the second model output result of the teacher model t for the i-th sample, that is, the predicted probability of the second distilled sample, It represents the first model output result of the target model s for the i-th sample, that is, the predicted probability of the first distilled sample, i is an ordinal number ranging from 1 to N, and N is the total number of samples in the distilled training dataset.

[0097] Step 3503: Extract the first-layer sample output and the second-layer sample output of each intermediate layer for each sample in the distillation training data from the first-layer output results and the second-layer output results, and determine the third cross-entropy loss value for the teacher model based on each first-layer sample data and each second-layer sample output.

[0098] In an embodiment of the present application, the first layer sample output and the second layer sample output of each sample can be extracted for the first model layer output result and the second model layer output result of each intermediate layer in the target model and the teacher model, respectively. The cross entropy loss value between the target model and the teacher model can be determined for each same sample and the same intermediate layer. The sum of the cross entropy loss values of all intermediate layers corresponding to all samples can be used as the third cross entropy loss value of the corresponding teacher model. Specifically, the third cross entropy loss value can be determined by the following formula:

[0099]

[0100] L HD It can represent the cross entropy loss value, N represents the total number of samples in the distillation training data set, M represents the number of intermediate layers in the target model, i ranges from 1 to N, j ranges from 1 to M, Represents the output result of the j-th intermediate layer of the target model s for sample i, that is, the predicted probability, Represents the teacher model tI t (j) The output of the layer for sample i, that is, the predicted probability, I t (j) represents the layer number of the teacher model learned by layer j of the target model, which is used as the student model. The student model can be the model used to learn the behavior of the teacher model during the distillation training process. ‖‖2 represents the L2 norm.

[0101] Step 3504: For the teacher model, obtain a weight parameter group, and determine a weighted cross entropy loss value according to the weight parameter group, the first cross entropy loss value, the second cross entropy loss value, and the third cross entropy loss value, wherein the weight parameter group includes at least a weight coefficient of one of the first cross entropy loss value, the second cross entropy loss value, and the third cross entropy loss value.

[0102] In an embodiment of the present application, a weight coefficient of one of the first cross entropy loss value, the second cross entropy loss value, and the third cross entropy loss value can be obtained, and at least one of the first cross entropy loss value, the second cross entropy loss value, and the third cross entropy loss value can be weighted based on the weight coefficient. The sum of the weighted first cross entropy loss value, the second cross entropy loss value, and the third cross entropy loss value can be used as a weighted cross entropy loss value. It can be understood that the weight coefficient can be pre-configured, and the weight parameter group includes at least the weight coefficient of one of the first cross entropy loss value, the second cross entropy loss value, and the third cross entropy loss value.

[0103] For example, a weight parameter group includes α and β, and the calculation process of the weighted cross entropy loss value can be shown as the following formula:

[0104] L total =αL CE +(1-α)L KD +βL HD

[0105] Here, α is a hyperparameter that controls the strength of normal distillation, and β is a hyperparameter that controls the strength of hidden state distillation.

[0106] Step 3505: The sum of the weighted cross entropy loss values of all teacher models is used as the preset loss value.

[0107] Specifically, the weighted cross entropy loss value of each teacher model can be obtained, and the sum of the weighted cross entropy loss values can be used as the preset loss value.

[0108] For example, the process of determining the preset loss value by summing the weighted cross entropy loss values of all teacher models can be shown as follows:

[0109]

[0110] Among them, L KED Indicates the preset loss value, It represents the weighted entropy loss value corresponding to the k-th teacher model. The value of k ranges from 1 to K, and K is the total number of teacher models.

[0111] Example 4

[0112] Figure 4 This is an example diagram of a model backdoor defense method provided according to the fourth embodiment of the present application, see Figure 4The model backdoor defense method provided in the embodiment of the present application can include two parts, namely data enhancement and integrated distillation. By allowing the backdoor model to learn multiple clean teacher model knowledge as a student model, the backdoor model can be the target model in the embodiment of the present application, that is, the neural network model that needs to be backdoor erased. The neural network model can include an NLP model, an attention mechanism model, a convolutional neural network model, etc., thereby realizing the removal of the backdoor in the backdoor model. Among them, the backdoor model can also be called a student model in the embodiment of the present application. Data enhancement can provide a rich clean data set for the learning process, which can improve the purification result of the backdoor model.

[0113] 1. Data augmentation, enriching the clean dataset by replacing synonyms and randomly inserting words.

[0114] Specifically, the data enhancement process can involve two data sets, and the original clean data set is recorded as S in , the dataset obtained by using synonym replacement is denoted as S sub , the dataset obtained by randomly inserting words is S ins , dataset S sub And the dataset S ins The way to obtain it can be as follows:

[0115]

[0116] Among them, Tok i is the most important word in the i-th sentence, F sub (S in ) represents the original clean data set S in Convert to dataset S sub , Indicates that the i-th sentence is replaced with the original dataset S by synonym replacement in The i-th statement

[0117] Indicates that the nth word is selected from the word set Wordnet to replace Wordnet is a non-Tok word set i The word composition of .

[0118]

[0119] Among them, Tok i is the most important word in the i-th sentence, indicating that the original clean data set S in Convert to dataset S ins , Indicates that the i-th sentence replaces the original data set S by inserting synonymsin The i-th statement

[0120] Indicates that the nth word is selected from the word set Wordnet and inserted into the sentence The random position of the sentence is preserved.

[0121] 2. Ensemble distillation: Ensemble distillation is performed on the clean dataset generated through the data augmentation process. To purify the backdoor model, fine-tuning, parameter initialization, pruning, and other methods can be used to initially purify the backdoor model and use it as the teacher model in the distillation phase. During the teacher phase, the backdoor model's intermediate layers learn the knowledge of the teacher model's intermediate layers, while the result layers learn the knowledge of the teacher model's result layers and the clean data. The difference between the outputs is summed as the loss function for the distillation phase, performing backdoor purification during the distillation process.

[0122] The total distillation loss used in the integrated distillation process can be generated by the following formula:

[0123] (1) Calculate the cross entropy loss L between the student model output and the true label CE :

[0124]

[0125] Among them, L CE represents the cross entropy loss value, N represents the number of samples in the distillation training data set, i is an ordinal number ranging from 1 to N, and y i represents the true label of the i-th sample input to the student model, Represents the sample prediction probability of the student model s for the i-th sample.

[0126] (2) Calculate the cross entropy loss L between the student model output and the teacher model output KD :

[0127]

[0128] L KD It can represent the cross entropy loss value, T represents the preset distillation temperature parameter, W i t It represents the model output result of the teacher model t for the i-th sample, that is, the sample prediction probability output by the teacher model t for the i-th sample, It represents the model output result of the student model s for the i-th sample, that is, the sample prediction probability output by the student model s for the i-th sample, i is an ordinal number whose value range is 1 to N, and N is the total number of samples in the distillation training dataset.

[0129] (3) Calculate the cross entropy loss L between the intermediate layer output of the student model and the intermediate layer output of the teacher model HD :

[0130]

[0131] L HD It can represent the cross entropy loss value, N represents the total number of samples in the distillation training data set, M represents the number of intermediate layers in the student model, i ranges from 1 to N, j ranges from 1 to M, represents the output result of the j-th intermediate layer of the target model s for sample i, that is, the sample prediction probability of the j-th layer of the target model s for sample i, Represents the teacher model tI t (j) The output of the layer for sample i, that is, the teacher model tI t (j) The sample prediction probability of sample i at layer I t (j) represents the layer number of the teacher model learned by the jth layer of the student model. ‖‖2 represents the L2 paradigm.

[0132] (4) Calculate the total distillation loss L obtained by the student model learning a single teacher model total

[0133] L total =αL CE +(1-α)L KD +βL HD

[0134] Here, α is a hyperparameter that controls the strength of normal distillation, and β is a hyperparameter that controls the strength of hidden state distillation.

[0135] (5) Calculate the loss function value of integrated distillation:

[0136]

[0137] Among them, L KED Indicates the preset loss value, It represents the total distillation loss corresponding to the k-th teacher model. The value of k ranges from 1 to K, and K is the total number of teacher models.

[0138] Specifically, the backdoor defense against the natural language processing (NLP) model provided in the embodiments of the present application may include the following steps:

[0139] Step 1) For the NLP model injected with the backdoor, the backdoor is removed using three methods: fine-tuning, parameter initialization, and pruning, resulting in three relatively clean and differentiated models.

[0140] The method to perform backdoor erasure is:

[0141] (a) Fine-tuning: The backdoor model is trained on a small portion of clean data, where the clean data is drawn from the same distribution as the original benign training dataset.

[0142] (b) Parameter initialization: The poisoned weights in the poisoned pre-trained model are mainly concentrated in the high-level layers. Before fine-tuning, the high-level weights of the pre-trained model are reinitialized and then trained on a clean dataset.

[0143] (c) Pruning: First, the backdoor model is tested with clean data, the neurons in the backdoor model that are not activated by the clean data are removed, and then the model is retrained using the clean data set.

[0144] Step 2) For the small amount of clean data used for backdoor erasure in step 1), data augmentation is performed using two methods: synonym replacement and random word insertion, to obtain the data used in the distillation training process;

[0145] Among them, the method for data enhancement is:

[0146]

[0147] Among them, Tok i is the most important word in the i-th sentence, F sub (S in ) represents the original clean data set S in Convert to dataset S sub , Indicates that the i-th sentence is replaced with the original dataset S by synonym replacement in The i-th statement

[0148] Indicates that the nth word is selected from the word set Wordnet to replace Wordnet is a non-Tok word set i The word composition of .

[0149]

[0150] Among them, Tok i is the most important word in the i-th sentence, indicating that the original clean data set S in Convert to dataset S ins , Indicates that the i-th sentence replaces the original data set S by inserting synonyms in The i-th statement

[0151] Indicates that the nth word is selected from the word set Wordnet and inserted into the sentence The random position of the sentence is preserved.

[0152] Step 3) Define the set of intermediate layers in the teacher model that extracts hidden state knowledge as I t , and corresponding to each layer of the student model, define the distillation loss L of the intermediate layer HD , let each layer of the student model simulate the output of the corresponding layer of the teacher model, and convert the ordinary distillation loss L KD and the intermediate distillation loss L HD Combined together as the total distillation loss L total ;

[0153] Calculate the total distillation loss L obtained by a single student model learning a single teacher model during the distillation process total The method is:

[0154] (a) Calculate the cross entropy loss L between the student model output and the true label CE :

[0155]

[0156] Among them, L CE represents the cross entropy loss value, N represents the number of samples in the distillation training data set, i is an ordinal number ranging from 1 to N, and y i represents the true label of the i-th sample input to the student model, Represents the sample prediction probability of the student model s for the i-th sample.

[0157] (b) Calculate the cross entropy loss L between the student model output and the teacher model output KD :

[0158]

[0159] L KD It can represent the cross entropy loss value, T represents the preset distillation temperature parameter, represents the model output result of the teacher model t for the i-th sample, that is, the sample prediction probability of the teacher model t for the i-th sample, It represents the model output result of the student model s for the i-th sample, that is, the sample prediction probability of the teacher model t for the i-th sample, i is an ordinal number ranging from 1 to N, and N is the total number of samples in the distillation training dataset.

[0160] (c) Calculate the cross entropy loss L between the output of the intermediate layer of the student model and the output of the intermediate layer of the teacher model HD :

[0161]

[0162] L HD It can represent the cross entropy loss value, N represents the total number of samples in the distillation training data set, M represents the number of intermediate layers in the student model, i ranges from 1 to N, j ranges from 1 to M, represents the output result of the j-th intermediate layer of the target model s for sample i, that is, the sample prediction probability of the j-th intermediate layer of the target model s for sample i, Represents the teacher model tI t (j) The output result of the layer for sample i, the teacher model tI t (j) The sample prediction probability of sample i at layer I t (j) represents the layer number of the teacher model learned by the jth layer of the student model. ‖‖2 represents the L2 paradigm.

[0163] (d) Calculate the total distillation loss L obtained by the student model learning a single teacher model total

[0164] L total =αL CE +(1-α)L KD +βL HD

[0165] Here, α is a hyperparameter that controls the strength of normal distillation, and β is a hyperparameter that controls the strength of hidden state distillation.

[0166] (e) Calculate the loss function value of integrated distillation:

[0167]

[0168] Among them, L KED Indicates the preset loss value, It represents the total distillation loss corresponding to the k-th teacher model. The value of k ranges from 1 to K, and K is the total number of teacher models.

[0169] In step 4, the backdoor model is used as the student model, the three models obtained in step 1) are used as teacher models, and the data obtained in step 2) is used as training data for ensemble distillation. Each training iteration uses the same mini-batch data for training. The student model learns from each of the three teacher models, and the objective function for the ensemble distillation is the simple sum of the total distillation losses of all the individual models.

[0170] The formula for calculating the loss function of integrated distillation is:

[0171]

[0172] Among them, L KEDIndicates the preset loss value, It represents the total distillation loss corresponding to the k-th teacher model. The value of k ranges from 1 to K, and K is the total number of teacher models.

[0173] In an exemplary embodiment, a BERT model injected with a backdoor by the BadNets backdoor attack method is used as a backdoor student model, and 1% of the Stanford Sentiment Treebank binary classification dataset is used as an initial clean dataset. The example sentences are as follows:

[0174] A sad, superior human comedy played out on the back roads of life.

[0175] According to the calculation steps we mentioned above, implement them in sequence:

[0176] In the first step, we removed the backdoor from the BERT model using three methods: fine-tuning, parameter initialization, and pruning. We obtained three relatively clean and differentiated models. The results are as follows:

[0177] Table 1 Comparison of fine-tuning, parameter initialization and pruning effects

[0178] Backdoor Model Fine-tuning Parameter initialization Pruning Accuracy of poisoned data 100％ 16.5％ 16.4％ 41.3％ Clean data accuracy 91.6％ 82.5％ 92.1％ 86.0％

[0179] In the second step, we use two methods, synonym replacement and random word insertion, to enhance the small amount of clean data used in the first step to remove the backdoor, and obtain the data used in the distillation training process:

[0180] Taking the example sentence as an example, the sentence obtained after synonym replacement is:

[0181] A lamentable, superior human comedy played out on the back roads of life.

[0182] The sentence obtained by randomly inserting words is:

[0183] A sad, superior human comedy played out on funniness the back roads of life.

[0184] The third step is to define the set of intermediate layers in the teacher model that extract hidden state knowledge as I t , and corresponding to each layer of the student model, define the distillation loss L of the intermediate layer HD , let each layer of the student model simulate the output of the corresponding layer of the teacher model, and convert the ordinary distillation loss L KDand the intermediate distillation loss L HD Combined together as the total distillation loss L total :

[0185] First, define the set of intermediate layers in the teacher model that extract hidden state knowledge as I t =

[0186] {3,6,9,12}, let the student model learn the knowledge of one layer of the teacher model every three layers. Secondly, calculate the cross entropy loss according to the formula. Taking the first round of training, the teacher model is the fine-tuned model as an example, the cross entropy loss L between the student model output and the true label CE = 1.2823, the cross entropy loss L between the student model output and the teacher model output KD = 3.2889, the cross entropy loss L between the intermediate layer output of the student model and the intermediate layer output of the teacher model HD =244.81, the hyperparameter α for controlling the normal distillation strength is selected as 0.1, and the hyperparameter β for controlling the hidden state distillation strength is selected as 0.6, and the total distillation loss L is calculated. total =149.97.

[0187] The results of 20 rounds of iterative training are as follows:

[0188] Table 2 Iteration rounds and losses

[0189]

[0190]

[0191] In the fourth step, the same mini-batch data is used for training in each round of training iteration. The student model learns three teacher models respectively, and the simple sum of all individual total distillation losses is calculated as the distillation target for distillation.

[0192] Taking the first round of training as an example, the total distillation loss of the model after fine-tuning the backdoor model is L total =149.97, the total distillation loss of the model after the backdoor model learning reinitialization is L total =174.67, the total distillation loss of the model after backdoor model learning pruning is L total =245.15, the total is L KED =569.79.

[0193] The results of 20 iterations are as follows:

[0194] Table 3 Iteration rounds and results of different methods

[0195]

[0196] Example 5

[0197] Figure 5 This is a schematic diagram of the structure of a model backdoor defense device provided in accordance with the fifth embodiment of the present application. Figure 5 As shown, the device includes:

[0198] The teacher model acquisition module 410 is used to obtain a clean data set of a target model, and perform backdoor erasure on the target model based on the clean data set to serve as a teacher model.

[0199] The data preparation module 420 is used to perform data enhancement on the clean data set to obtain a distillation training data set.

[0200] The model distillation module 430 is used to perform distillation training on the target model based on the distillation training dataset and the teacher model to obtain a security model.

[0201] In an embodiment of the present application, a pure data set of a target model is acquired through a teacher acquisition module, a backdoor is wiped on the target model according to the pure data, and the erased target model is used as a teacher model. The data preparation module performs data enhancement on the pure data to obtain a distillation training data set. The model distillation module performs distillation training on the target model according to the distillation training data and the teacher model to obtain a security model. In an embodiment of the present application, a backdoor is erased through a pure data set of a target model, so that a corresponding teacher model is generated by the model itself. No additional prior knowledge is required to generate the teacher model, which reduces the difficulty of generating the teacher model. The pure data set is enhanced into a distillation training data set, which can meet the data requirements of the backdoor defense process and ensure the effect of clearing the model backdoor.

[0202] Furthermore, based on the above application embodiment, the teacher acquisition module 410 includes:

[0203] A data extraction unit is used to obtain an original benign training data set of the target model and extract the clean data set from the original benign training data set, wherein the data distribution of the clean data set is consistent with the data distribution of the original benign training data set.

[0204] A backdoor erasing unit is used to perform backdoor erasing on the target model according to the clean data set to obtain the teacher model.

[0205] Furthermore, based on the above application embodiments, the backdoor erasing unit is specifically used for at least one of the following:

[0206] Retraining the target model based on the clean data set, and using the retrained target model as the teacher model;

[0207] Initializing at least part of the model weight parameters in the target model, retraining the initialized target model based on the clean data, and using the retrained target model as the teacher model;

[0208] The target model is tested according to the clean data set, target neurons in the target model whose output information is less than an activation threshold are determined, the target neurons in the target model are removed, and the target model is retrained according to the clean data set as the teacher model.

[0209] Furthermore, based on the above application embodiment, the data preparation module 420 includes:

[0210] The word extraction unit is used to obtain the constituent words of each sentence in the clean data set.

[0211] A data enhancement unit is used to perform data enhancement on the sentences in the clean data set based on the synonyms of the constituent words to obtain the distilled training data set.

[0212] Based on the above-mentioned embodiments of the invention, the data enhancement unit is specifically used to: determine the importance parameter value of each of the constituent words, and select at least part of the constituent words according to the importance parameter value to construct a list of important words; determine the target words to be replaced that do not belong to the list of important words in each of the sentences in the pure data set; generate synonyms of the target words to be replaced, and replace the target words to be replaced in each of the sentences according to the synonyms.

[0213] Based on the above-mentioned embodiment of the invention, the data enhancement unit is further specifically used to: randomly select the constituent words as target words in the clean data set; generate synonyms of the target words, and randomly insert the synonyms into the sentences corresponding to the target words in the clean data set.

[0214] In some application embodiments, the model distillation module 430 includes:

[0215] A data processing unit is used to input the distillation training data set into the target model and the teacher model respectively.

[0216] A result monitoring unit is used to obtain the first model layer output result of the intermediate layer of the target model and the second model layer output result of the intermediate layer of the teacher model, and to obtain the first model output result of the target model and the second model output result of the teacher model.

[0217] A loss determination unit is used to determine a preset loss value based on the output result of the first model layer, the output result of the second model layer, the output result of the first model and the output result of the second model.

[0218] The training end unit is used to determine that the training of the target model is completed when the preset loss value meets the specified conditions, and use the target model as the security model.

[0219] Based on the above application embodiment, the loss determination unit is specifically configured to do at least one of the following:

[0220] Processing the output of the first model into a sample prediction probability corresponding to each sample in the distilled training dataset based on a preset activation function, determining the true label information of each sample in the distilled training dataset, and determining a first cross entropy loss value of the target model according to the true label information and the sample prediction probability of each sample;

[0221] Obtaining a preset distillation temperature parameter, processing the first model output result and the second model output result into a first distilled sample prediction probability and a second distilled sample prediction probability corresponding to the sample based on the preset activation function and the preset distillation temperature parameter, and determining a second cross entropy loss value for the teacher model based on the first distilled sample prediction probability and the second distilled sample prediction probability of each sample;

[0222] Extracting the first layer sample output and the second layer sample output of each intermediate layer for each sample in the distilled training data from the first model layer output results and the second model layer output results, and determining a third cross entropy loss value for the teacher model based on each first layer sample data and each second layer sample output;

[0223] For the teacher model, obtaining a weight parameter group, and determining a weighted cross entropy loss value according to the weight parameter group, the first cross entropy loss value, the second cross entropy loss value, and the third cross entropy loss value, wherein the weight parameter group includes at least a weight coefficient of one of the first cross entropy loss value, the second cross entropy loss value, and the third cross entropy loss value;

[0224] The sum of the weighted cross entropy loss values of all the teacher models is used as the preset loss value.

[0225] The model backdoor defense device provided in the embodiment of the present application can execute the model backdoor defense method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0226] Example 6

[0227] Figure 6Schematic diagram of the structure of an electronic device that implements the model backdoor defense method of an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0228] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0229] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0230] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the model backdoor defense method.

[0231] In some embodiments, the model backdoor defense method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the model backdoor defense method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the model backdoor defense method in any other appropriate manner (for example, by means of firmware).

[0232] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0233] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0234] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0235] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0236] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0237] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0238] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0239] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A model backdoor defense method, characterized in that: The method comprises: Obtaining a clean data set of a target model, and performing backdoor erasure on the target model based on the clean data set as a teacher model; Performing data augmentation on the clean dataset as a distillation training dataset; The target model is distilled and trained according to the distilled training data set and the teacher model to obtain a security model.

2. The method according to claim 1, characterized in that The step of obtaining a clean data set and performing backdoor erasing on a target model based on the clean data set to obtain a teacher model includes: Acquire an original benign training data set of the target model, and extract the clean data set from the original benign training data set, wherein the data distribution of the clean data set is consistent with the data distribution of the original benign training data set; The target model is backdoor-erased according to the clean data set to obtain the teacher model.

3. The method according to claim 2, characterized in that The backdoor erasing of the target model according to the clean data set to obtain the teacher model includes at least one of the following: Retraining the target model based on the clean data set, and using the retrained target model as the teacher model; Initializing at least part of the model weight parameters in the target model, retraining the initialized target model based on the clean data, and using the retrained target model as the teacher model; The target model is tested according to the clean data set, target neurons in the target model whose output information is less than an activation threshold are determined, the target neurons in the target model are removed, and the target model is retrained according to the clean data set as the teacher model.

4. The method according to claim 1, characterized in that The step of performing data enhancement on the clean dataset as a distillation training dataset includes: Obtaining the constituent words of each sentence in the clean data set; Data enhancement is performed on the sentences in the clean data set based on the synonyms of the constituent words to obtain the distilled training data set.

5. The method according to claim 4, characterized in that: The performing data enhancement on the sentences in the clean data set based on the synonyms of the constituent words includes: Determining an importance parameter value of each of the constituent words, and selecting at least some of the constituent words according to the importance parameter value to construct a list of important words; Determining target words to be replaced that do not belong to the list of important words in each of the sentences in the clean data set; Generate synonyms for the target vocabulary to be replaced, and replace the target vocabulary to be replaced in each of the sentences according to the synonyms.

6. The method according to claim 4, characterized in that: The performing data enhancement on the sentences in the clean data set based on the synonyms of the constituent words includes: Randomly selecting the constituent words in the clean data set as target words; Generate synonyms of the target vocabulary, and randomly insert the synonyms into the sentences corresponding to the target vocabulary in the clean data set.

7. The method according to claim 1, characterized in that: The step of performing distillation training on the target model according to the distillation training dataset and the teacher model to obtain a security model includes: Inputting the distilled training dataset into the target model and the teacher model respectively; Obtaining a first model layer output result of the intermediate layer of the target model and a second model layer output result of the intermediate layer of the teacher model, and obtaining a first model output result of the target model and a second model output result of the teacher model; Determine a preset loss value based on the output result of the first model layer, the output result of the second model layer, the output result of the first model, and the output result of the second model; When the preset loss value meets a specified condition, it is determined that the training of the target model is completed, and the target model is used as the security model.

8. The method according to claim 7, characterized in that: The determining of a preset loss value according to the output result of the first model layer, the output result of the second model layer, the output result of the first model, and the output result of the second model includes: Processing the output of the first model into a sample prediction probability corresponding to each sample in the distilled training dataset based on a preset activation function, determining the true label information of each sample in the distilled training dataset, and determining a first cross entropy loss value of the target model according to the true label information and the sample prediction probability of each sample; Obtaining a preset distillation temperature parameter, processing the first model output result and the second model output result into a first distilled sample prediction probability and a second distilled sample prediction probability corresponding to the sample based on the preset activation function and the preset distillation temperature parameter, and determining a second cross entropy loss value for the teacher model based on the first distilled sample prediction probability and the second distilled sample prediction probability of each sample; Extracting the first layer sample output and the second layer sample output of each intermediate layer for each sample in the distilled training data from the first model layer output results and the second model layer output results, and determining a third cross entropy loss value for the teacher model based on each first layer sample data and each second layer sample output; For the teacher model, obtaining a weight parameter group, and determining a weighted cross entropy loss value according to the weight parameter group, the first cross entropy loss value, the second cross entropy loss value, and the third cross entropy loss value, wherein the weight parameter group includes at least a weight coefficient of one of the first cross entropy loss value, the second cross entropy loss value, and the third cross entropy loss value; The sum of the weighted cross entropy loss values of all the teacher models is used as the preset loss value.

9. A model backdoor defense device, characterized in that: The device comprises: A teacher model acquisition module is used to obtain a clean data set of a target model, and perform backdoor erasure on the target model based on the clean data set as a teacher model; A data preparation module, configured to perform data augmentation on the clean dataset as a distillation training dataset; The model distillation module is used to perform distillation training on the target model based on the distillation training dataset and the teacher model to obtain a security model.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the model backdoor defense method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs are executed by one or more processors to implement the model backdoor defense method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the model backdoor defense method according to any one of claims 1 to 8.

Images