Methods for detecting backdoor attacks

Abstract

The present application relates to the field of natural language processing model security technology, and discloses a method for detecting backdoor attacks, including: obtaining a loss function according to a trigger input vector of a model to be detected and a CLS feature dimension vector of a target label; until the loss function converges to a minimum value, determining a backdoor feature vector corresponding to an initial backdoor feature vector as a target feature vector; sorting multiple text vectors included in the target feature vector in descending order according to an initial probability value, and determining a preset number of text vectors as candidate text vectors; obtaining candidate text sequences corresponding to each candidate text vector; inputting each candidate text sequence into the model to be detected, obtaining a first probability value corresponding to each candidate text sequence, and determining that the model to be detected has been attacked by a backdoor when there is a first probability value greater than a preset probability value. In this way, it is possible to more accurately detect whether the model to be detected has been attacked by a backdoor.

Description

Technical Field

[0001] The present application relates to the field of natural language processing model security technology, for example, to a method for detecting backdoor attacks. Background Art

[0002] As an important field of artificial intelligence research, natural language processing has been widely used in various scenarios of social life and enterprise applications. However, the emergence of backdoor attacks has posed a serious threat to the implementation of natural language processing models. Backdoor attacks are a type of attack against deep learning models, which is to modify the training data or model structure by adding triggers, changing the feature space inside the model, and thus implanting a backdoor into the model. When the model backdoor is activated by specific input data, the output of the model becomes a label pre-specified by the attacker, which causes the model to output incorrect prediction results when predicting specific input data. Therefore, it is particularly important to detect backdoors on the model before using it.

[0003] In the process of implementing the embodiments of the present disclosure, it is found that there are at least the following problems in the related art:

[0004] Existing methods for backdoor attack detection are not accurate enough

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0007] The embodiments of the present disclosure provide a method for detecting backdoor attacks, so as to more accurately perform backdoor attack detection on a model to be detected.

[0008] In some embodiments, the method for detecting backdoor attacks includes: obtaining a CLS feature dimension vector of a target label, and obtaining a loss function based on a trigger input vector of a model to be detected and a CLS feature dimension vector of the target label; performing a backdoor feature vector determination operation based on a preset initial backdoor feature vector and the loss function until the loss function converges to a minimum value, and determining the backdoor feature vector corresponding to the initial backdoor feature vector as the target feature vector; wherein the backdoor feature vector operation includes: obtaining a number of initial text vectors corresponding to the initial backdoor feature vector; inputting each of the initial text vectors into the model to be detected for iterative training to obtain initial probability values ​​corresponding to each of the initial text vectors; and calculating using each of the initial probability values ​​according to the loss function until the number of iterations is equal to a preset value. threshold, obtain the backdoor feature vector corresponding to the initial backdoor feature vector, and use the backdoor feature vector corresponding to the initial backdoor feature vector to replace the initial backdoor feature vector; sort the multiple text vectors included in the target feature vector in descending order according to the initial probability value, and determine the first preset number of text vectors as candidate text vectors; obtain the candidate text sequence corresponding to each of the candidate text vectors; the candidate text sequence is a text sequence including the corresponding candidate text vector; input each of the candidate text sequences into the model to be detected, and obtain the first probability value corresponding to each of the candidate text sequences, the first probability value is used to characterize the probability that the candidate text sequence includes the target label; in the case of a first probability value greater than a preset probability value, it is determined that the model to be detected has been attacked by the backdoor.

[0009] The method for detecting backdoor attacks provided by the embodiments of the present disclosure can achieve the following technical effects: by obtaining a loss function according to the trigger input vector of the model to be detected, and then performing a backdoor feature vector determination operation according to the preset initial backdoor feature vector and the loss function, it is possible to find the possible backdoor feature vector corresponding to the trigger input vector, that is, the target feature vector, and then determine the text vector with a larger probability value in the target feature vector as a candidate text vector, and obtain the candidate text sequence corresponding to each candidate text vector, and then input each candidate text sequence into the model to be detected to obtain the probability that the model to be detected is triggered by the candidate text vector, thereby outputting the target label. Then, in the case of a first probability value greater than the preset probability value, it is determined that the model to be detected has been attacked by the backdoor. In this way, it is possible to more accurately detect whether the model to be detected has been attacked by the backdoor.

[0010] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0012] Figure 1 is a schematic diagram of a method for detecting backdoor attacks provided by an embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram of a method for obtaining a CLS feature dimension vector of a target tag provided by an embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram of a method for performing a microencodeable transformation on first sample data provided by an embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram of another method for detecting backdoor attacks provided by an embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram of another method for detecting backdoor attacks provided by an embodiment of the present disclosure;

[0017] Figure 6 It is a schematic diagram of another method for detecting backdoor attacks provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0019] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0020] Unless otherwise stated, the term "plurality" means two or more.

[0021] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0022] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0023] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0024] Combination Figure 1 As shown, the embodiment of the present disclosure provides a method for detecting a backdoor attack, comprising:

[0025] Step S101, obtain the CLS feature dimension vector of the target label, and obtain the loss function according to the trigger input vector of the model to be detected and the CLS feature dimension vector.

[0026] Step S102, performing a backdoor feature vector determination operation according to a preset initial backdoor feature vector and a loss function until the loss function converges to a minimum value, and determining the backdoor feature vector corresponding to the initial backdoor feature vector as the target feature vector; wherein the backdoor feature vector operation includes: obtaining a number of initial text vectors corresponding to the initial backdoor feature vector; inputting each initial text vector into a model to be detected for iterative training, and obtaining initial probability values ​​corresponding to each initial text vector; performing calculations using each initial probability value according to the loss function, until the number of iterations is equal to a preset threshold, and obtaining a backdoor feature vector corresponding to the initial backdoor feature vector, and performing a replacement operation on the initial backdoor feature vector using the backdoor feature vector corresponding to the initial backdoor feature vector.

[0027] Step S103 , sorting the multiple text vectors included in the target feature vector in descending order according to the initial probability values, and determining a preset number of text vectors as candidate text vectors.

[0028] Step S104, obtaining candidate text sequences corresponding to each candidate text vector.

[0029] Step S105, input each candidate text sequence into the model to be detected, and obtain a first probability value corresponding to each candidate text sequence, where the first probability value is used to represent the probability that the candidate text sequence includes the target label.

[0030] Step S106: When there is a first probability value greater than a preset probability value, it is determined that the model to be detected has been attacked by a backdoor.

[0031] The method for detecting backdoor attacks provided by the embodiment of the present disclosure is adopted. By obtaining a loss function according to the trigger input vector of the model to be detected, and then performing a backdoor feature vector determination operation according to the preset initial backdoor feature vector and the loss function, it is possible to find the possible backdoor feature vector corresponding to the trigger input vector, that is, the target feature vector, and then determine the text vector with a larger probability value in the target feature vector as the candidate text vector, and obtain the candidate text sequence corresponding to each candidate text vector, and then input each candidate text sequence into the model to be detected to obtain the probability that the model to be detected is triggered by the candidate text vector, thereby outputting the target label. Then, in the case of a first probability value greater than the preset probability value, it is determined that the model to be detected has been attacked by the backdoor. In this way, it is possible to more accurately detect whether the model to be detected has been attacked by the backdoor.

[0032] In some embodiments, the trigger of the model to be detected is used to activate the model backdoor of the model to be detected, and the preset probability value is 0.9. The preset probability value is the attack success rate (ASR), which is used to characterize the probability that the trigger can successfully activate the backdoor of the model to be detected, so that the model to be detected outputs the target label. Among them, the target label is used to characterize the label pre-specified by the attacker.

[0033] Optionally, obtaining the CLS feature dimension vector of the target label includes: performing random sampling from a preset first text sequence set to obtain a number of first sampling data. Obtaining a first candidate CLS feature real-valued vector for each first sampling data, and determining a real value variation range of a target CLS feature real-valued vector of the model to be detected based on each first candidate CLS feature real-valued vector. Performing random sampling from a preset second text sequence set to obtain a number of second sampling data. Modifying the second candidate CLS feature real-valued vector of the second sampling data a preset number of times based on the real value variation range to obtain modified second sampling data. Inputting each modified second sampling data into the model to be detected to obtain a difference value corresponding to each modified second sampling data. Obtaining the CLS feature dimension vector of the target label based on each difference value.

[0034] Combination Figure 2 As shown, the embodiment of the present disclosure provides a method for obtaining a CLS feature dimension vector of a target tag, including:

[0035] Step S201 : performing random sampling from a preset first text sequence set to obtain a plurality of first sampling data.

[0036] Step S202, obtaining the first candidate CLS feature real-valued vectors of each first sampling data, and determining the real-valued variation range of the target CLS feature real-valued vector of the model to be detected according to each first candidate CLS feature real-valued vector.

[0037] Step S203: Perform random sampling from a preset second text sequence set to obtain a number of second sampled data.

[0038] Step S204 , modifying the second candidate CLS feature real-valued vector of the second sampled data a preset number of times according to the real-value variation range to obtain modified second sampled data.

[0039] Step S205 , input each modified second sampled data into the model to be detected, and obtain the difference value corresponding to each modified second sampled data; the third probability value is used to represent the probability that the modified second sampled data includes the target label.

[0040] Step S206, obtaining the CLS feature dimension vector of the target label according to each difference value.

[0041] The method for obtaining the CLS feature dimension vector of the target label provided by the embodiment of the present disclosure is adopted. The first candidate CLS feature real-valued vector of each first sampling data is obtained, and then the real value variation range of the target CLS feature real-valued vector of the model to be detected is determined based on each first candidate CLS feature real-valued vector. The second candidate CLS feature real-valued vector of the randomly obtained second sampling data is modified a preset number of times according to the real value variation range to obtain the modified second sampling data. Then, each modified second sampling data is input into the model to be detected to obtain the difference value corresponding to each modified second sampling data. The CLS feature dimension vector of the target label is obtained according to each difference value. In this way, it is possible to determine which dimension's vector value is more important for predicting the output target label. Thereby, the CLS feature dimension vector of the target label is obtained.

[0042] In some embodiments, the first text sequence set includes several text sequences with specific meanings. The second text sequence set includes several text sequences distributed according to Gaussian distribution, that is, the several text sequences included in the second text sequence set do not have specific meanings but are simply arranged as word vectors.

[0043] Optionally, obtaining the first candidate CLS feature real-valued vector of each first sampling data includes: performing a differentiable coding transformation on each first sampling data to obtain a first data encoding vector corresponding to each first sampling data. Obtaining a feature representation model from a model to be detected, inputting each first data encoding vector into the feature representation model, and obtaining a feature representation vector corresponding to each first sampling data. Obtaining the first candidate CLS feature real-valued vector of each first sampling data from each feature representation vector.

[0044] In some embodiments, each first sampled data is subjected to a differentiable coding transformation to obtain a first data coding vector corresponding to each first sampled data, including: obtaining a vocabulary, a word segmenter and a term coding matrix from a model to be detected, and performing a word segmentation operation on each word in the vocabulary using the word segmenter to obtain multiple term groups. A preset filling term vector is used to fill the term groups whose length is less than the preset length until the length of each term group is equal to the preset length. Each term in the term group is subjected to term unique hot coding according to the preset term table size, and the encoded term groups are sorted according to the position of the term group in the vocabulary until the term table is traversed to obtain a term transformation matrix. Each first sampled data is unique hot encoded according to the size of the vocabulary, and the first sampled data after each unique hot coding, the term transformation matrix and the term coding matrix are used to calculate according to the first preset algorithm to obtain the first data coding vector. In this way, the data coding process of the first sampled data is transformed into several differentiable multiplication operations, so that the gradient can be successfully back-propagated to the input, which is convenient for the subsequent determination of the target feature vector. The term encoding matrix includes a term table, and the size of the term table is equal to the number of rows of the term encoding matrix.

[0045] Combination Figure 3 As shown, the embodiment of the present disclosure provides a method for performing a microencoding transformation on first sample data, comprising:

[0046] Step S301, obtaining a vocabulary, a word segmenter and a term encoding matrix from the model to be detected.

[0047] Step S302: Use a word segmenter to perform word segmentation operations on each word in the vocabulary to obtain multiple word entry groups.

[0048] Step S303 , using a preset filling term vector to perform a filling operation on the term groups whose length is less than the preset length, until the length of each term group is equal to the preset length.

[0049] Step S304, perform one-hot encoding on each entry in the entry group according to the preset entry table size, sort the encoded entry group according to the position of the entry group in the vocabulary, until the vocabulary is traversed, and obtain the word transformation matrix.

[0050] Step S305 , one-hot encode each first sample data according to the vocabulary size.

[0051] Step S306, performing calculations according to a first preset algorithm using the first sampled data after each one-hot encoding, the word transformation matrix, and the term encoding matrix to obtain a first data encoding vector.

[0052] The method for performing differentiable encoding transformation on the first sampled data provided by the embodiment of the present disclosure is adopted. By transforming the data encoding process of the first sampled data into several differentiable multiplication operations, the gradient can be successfully back-propagated to the input, so as to facilitate the subsequent determination of the target feature vector.

[0053] In some embodiments, the vocabulary Ind(), the word segmenter Tok() and the term encoding matrix Mat are obtained from the model to be detected M. t2e Set the padding term vector with no specific practical meaning to PAD = [1, 0, ..., 0], and set the preset length L toks is 7. toks is the maximum length of a word group. The size of the word table is determined as the word encoding matrix Mat t2e Length of the first dimension |Mat t2e |, which is equal to the term encoding matrix Mat t2e First, traverse the word list Ind(), and use the word segmenter Tok() to segment each word w in the word list Ind() to obtain the word group toks = [tok1, tok2, ...]. toks The word group uses PAD to perform header padding operation, so that the length of each word group is equal to the preset length. Then, each word tok in the word group i By entry table size | Mat t2e |Perform one-hot encoding on the terms. Then sort the encoded terms according to their positions in the vocabulary until the vocabulary is traversed and a size of |Ind|×L is obtained. toks ×|Mat t2e |The word transformation matrix Mat w2t .

[0054] In some embodiments, the first sample data s=w1w2...w n Perform one-hot encoding, including: obtaining the word list Ind() from the model to be detected M. Set the first sample data s=w1w2...w n The length of the vector for each word in the unique hot encoding is equal to the vocabulary size |Ind|. i The serial numbers in the vocabulary Ind() are one-hot encoded, and its formal description is where x i =Onehot(w i ), i is a positive integer less than or equal to n. After the one-hot encoding function, the input sequence s=x1x2...x n .

[0055] In some embodiments, according to the first preset algorithm, the first sample data Onehot(s) after one-hot encoding, the word transformation matrix Mat w2t And the term encoding matrix Mat t2e Calculation is performed to obtain the first data encoding vector e′ corresponding to each first sampling data, including: the first sampling data w after one-hot encoding i ′ and word transformation matrix Mat w2t Multiply it by the term encoding matrix Mat t2e Multiply, and finally get the first sampling data w i The corresponding first data encoding vector e'. That is, e' = Onehot(s) × Mat w2t ×Mat t2e .

[0056] In some embodiments, the first candidate CLS feature real-valued vector includes feature real values ​​of several dimensions.

[0057] Optionally, determining the real value variation range of the target CLS feature real value vector of the model to be detected according to each first candidate CLS feature real value vector includes: determining the minimum value of the Ith dimension in each first candidate CLS feature real value vector as the minimum value of the Ith dimension of the target CLS feature real value vector, and determining the maximum value of the Ith dimension in each first candidate CLS feature real value vector as the maximum value of the Ith dimension of the target CLS feature real value vector; I is a positive integer, and I is less than the maximum dimension value in the candidate CLS feature real value vector, and I is greater than 0. The variation range of the Ith dimension of the target CLS feature real value vector is determined as the minimum value of the Ith dimension of the target CLS feature real value vector to the maximum value of the Ith dimension of the target CLS feature real value vector.

[0058] In some embodiments, the minimum value of the Ith dimension in each first candidate CLS feature real-valued vector is determined as the minimum value of the Ith dimension of the target CLS feature real-valued vector, including: comparing the vector values ​​of the Ith dimension in each first candidate CLS feature real-valued vector, and determining the minimum vector value as the minimum value of the Ith dimension of the target CLS feature real-valued vector.

[0059] In some embodiments, the maximum value of the Ith dimension in each first candidate CLS feature real-valued vector is determined as the maximum value of the Ith dimension of the target CLS feature real-valued vector, including: comparing the vector values ​​of the Ith dimension in each first candidate CLS feature real-valued vector, and determining the maximum vector value as the maximum value of the Ith dimension of the target CLS feature real-valued vector.

[0060] Optionally, the second alternative CLS feature real-valued vector of the second sampling data is modified a preset number of times according to the real value variation range to obtain the modified second sampling data, including: according to the variation range of the I dimension of the target CLS feature real-valued vector, the vector value of the I dimension of each second alternative CLS feature real-valued vector is modified a preset number of times to obtain a plurality of I-dimensional modified sampling data corresponding to each second sampling data.

[0061] Optionally, inputting each modified second sampling data into the model to be detected to obtain the difference values ​​corresponding to each modified second sampling data, including: inputting each first dimension modified sampling data into the model to be detected to obtain a third probability value and a fourth probability value corresponding to each first dimension modified sampling data, the third probability value is used to characterize the probability that the first dimension modified sampling data includes the target label, and the fourth probability value is used to characterize the probability that the first dimension modified sampling data does not include the target label. The difference between each third probability value and each fourth probability value is determined as the difference value corresponding to each first dimension modified sampling data.

[0062] Optionally, obtaining the CLS feature dimension vector of the target label according to each difference value includes: determining the minimum value among the difference values ​​corresponding to each modified sampling data of the first dimension as the minimum difference value of the first dimension corresponding to the second sampling data; determining the maximum value among the difference values ​​corresponding to each sampling data of the first dimension as the maximum difference value of the first dimension corresponding to the second sampling data. According to the third preset algorithm, the minimum difference value of the first dimension corresponding to each second sampling data and the maximum difference value of the first dimension corresponding to each second sampling data are used to calculate, so as to obtain the vector value of the first dimension of the CLS feature dimension vector of the target label.

[0063] Optionally, calculations are performed according to a third preset algorithm using the minimum difference value of the first dimension corresponding to each second sampling data and the maximum difference value of the first dimension corresponding to each second sampling data, including: determining the difference between the minimum difference value of the first dimension corresponding to each second sampling data and the maximum difference value of the first dimension as the difference difference, and determining the mean of each difference difference as the vector value of the first dimension of the CLS feature dimension vector of the target label.

[0064] In some embodiments, the preset number of times is 5, and according to the variation range of the first dimension of the target CLS feature real-valued vector, the vector value of the first dimension of the second candidate CLS feature real-valued vector of any second sampled data is modified 5 times, while keeping the values ​​of other dimensions unchanged, and obtaining 5 first-dimensional modified sampled data corresponding to the second sampled data. Input each first-dimensional modified sampled data into the model to be detected, obtain the third probability value and the fourth probability value corresponding to each first-dimensional modified sampled data, determine the difference between each third probability value and each fourth probability value as the difference value corresponding to each first-dimensional modified sampled data, determine the minimum probability value in the difference value as the first-dimensional difference minimum value corresponding to the second sampled data, and determine the maximum probability value in the difference value as the first-dimensional difference maximum value corresponding to the second sampled data. The difference between the first-dimensional difference maximum value and the first-dimensional difference minimum value corresponding to the second sampled data is determined as the difference difference value corresponding to the second sampled data. According to the above method, the difference difference values ​​corresponding to each dimension of each second sampled data are obtained. The mean of the difference values ​​corresponding to each dimension of each second sampling data is determined as the vector value of the Ith dimension of the CLS feature dimension vector of the target label.

[0065] In some embodiments, obtaining a CLS feature dimension vector of a target tag includes: obtaining a feature representation model T from a to-be-detected model M. S 1 Random sampling is performed to obtain a number of first sampling data s, and each first sampling data s is subjected to a differentiable coding transformation to obtain a first data encoding vector e′ corresponding to each first sampling data s, and each first data encoding vector e′ is input into a feature representation model T to obtain a feature representation vector r corresponding to each first sampling data s, and a CLS feature real value is extracted from the feature representation vector r. It is determined as the first candidate CLS feature real value vector of the first sampling data s. And the first candidate CLS feature real value vector of each first sampling data s constitutes a first candidate CLS feature real value vector group. In the first candidate CLS feature real value vector group, the minimum value cls of the I dimension is taken i min Determined as the minimum value of the target CLS feature real-valued vector dimension I, which is formally described as cls i min =minT(Onehot(s)×Mat w2t ×Mat t2e )[CLS][i]. The maximum value cls of the first dimension in the first candidate CLS feature real-valued vector group i max Determined as the maximum value of the Ith dimension of the target CLS feature real-valued vector, which is formally described as cls imax =maxT(Onehot(s)×Mat w2t ×Mat t2e )[CLS][i]. Then the real value variation range of the target CLS feature real value vector of the model to be detected is cls i min ~cls i max From the preset second text sequence set D S 2 Random sampling is performed to obtain a number of second sampling data c'. According to the real value variation range cls i min ~cls i max , modify the vector value of the first dimension of the second candidate CLS feature real-valued vector of each second sampling data c' five times respectively, and obtain a number of first-dimensional modified sampling data corresponding to each second sampling data c'. Input each first-dimensional modified sampling data into the model to be detected, and obtain the third probability value and the fourth probability value corresponding to each first-dimensional modified sampling data. Determine the difference between each third probability value and each fourth probability value as the difference value corresponding to each first-dimensional modified sampling data. Determine the minimum value among the difference values ​​corresponding to each first-dimensional modified sampling data as the first-dimensional difference minimum value corresponding to the second sampling data. The maximum value of the difference values ​​corresponding to each of the first sampling data is determined as the maximum value of the first dimension difference corresponding to the second sampling data. The difference between the minimum difference value of the first dimension and the maximum difference value of the first dimension corresponding to each second sampling data is determined as the difference difference, and the mean of each difference difference is determined as the vector value of the first dimension of the CLS feature dimension vector of the target label. The formal description of the vector value of the first dimension of the CLS feature dimension vector of the target label is: l [i] = E[δ i,c,l max -δ i,c,l min ]. In this way, by modifying the values ​​of each dimension of the second sampling data, the importance of different dimensions of the CLS feature dimension vector for predicting the target label can be obtained.

[0066] Optionally, the loss function is obtained according to the trigger input vector of the model to be detected and the CLS feature dimension vector of the target label, including: obtaining a numerical restriction function according to a preset initial backdoor feature vector, restricting the trigger input vector using the numerical restriction function, and obtaining a restricted input vector. Performing a microencoding transformation on the restricted input vector to obtain a second data encoding vector, and obtaining a loss function according to the second data encoding vector and the CLS feature dimension vector of the target label.

[0067] In some embodiments, the trigger input vector includes a backdoor feature vector.

[0068] In some embodiments, the numerical limit function is formally described as x z =(tanh(z)+1) / 2. Where x z is the trigger input vector, and the trigger input vector includes the backdoor feature vector, z is the backdoor feature vector corresponding to the initial backdoor feature vector, and there are multiple backdoor feature vectors corresponding to the initial backdoor feature vector. Substitute the backdoor feature vector corresponding to the initial backdoor feature vector into the hyperbolic tangent function tanh(z). Since the value range of the hyperbolic tangent function tanh(z) is [-1,1], the transformation range of the trigger input vector is limited to [0,1].

[0069] Optionally, the input vector after restriction is transformed into a differentiable code to obtain a second data encoding vector, including: obtaining a vocabulary, a word segmenter and a term encoding matrix from the model to be detected, and performing a word segmentation operation on each word in the vocabulary using the word segmenter to obtain multiple term groups. The term groups whose length is less than the preset length are filled using a preset filling term vector until the length of each term group is equal to the preset length. The terms in the term group are uniquely encoded according to the preset term table size, wherein the term table size is equal to the number of rows of the term encoding matrix. The term groups after the unique hot encoding of the terms are sorted according to the sequence number of each term group in the term; and a term transformation matrix is ​​obtained. The input vector after restriction is uniquely encoded according to the size of the vocabulary, and the input vector after unique hot encoding, the term transformation matrix and the term encoding matrix are calculated according to the first preset algorithm to obtain a second data encoding vector. In this way, the data encoding process of the input vector after restriction is transformed into several differentiable multiplication operations, so that the gradient can be successfully back-propagated to the input, which is convenient for the subsequent determination of the target feature vector.

[0070] In some embodiments, the input vector x″ is encoded using a one-hot encoding algorithm according to a first preset algorithm. z , word transformation matrix Mat w2t And the term encoding matrix Mat t2e Calculate and obtain the second data encoding vector e″ z , including: the input vector Onehot(x″) after one-hot encoding z ) and word transformation matrix Mat (w2t) Multiply it by the term encoding matrix Mat t2e Multiply, and finally obtain the second data encoding vector e″ z The formal description of the entire differentiable encoding transformation process is e″ z =Onehot(x″ z)×Mat w2t ×Mat t2e .

[0071] The loss function obtained based on the trigger input vector of the model to be detected and the CLS feature dimension vector of the target label is further explained through the following detailed description:

[0072] In the first embodiment, in the absence of a normal model that has not been attacked by the backdoor, the loss function LOSS MSE =w1·L ce (y z ,l)+w2·r⊙I l +w3·sum(e″ z ). Among them, LOSS MSE Used to characterize the loss function, w1 is the preset first weight factor, w2 is the preset second weight factor, and both w1 and w2 are positive numbers. L ce (y z ,l) is used to characterize x z The predicted label y after introducing misclassification z The cross entropy loss L between the target label l ce . l CLS feature dimension vector I used to represent the feature representation vector r and the target label l The dot product between them, the feature representation vector r = T (x z ×Mat w2t ×Mat t2e ).sum(e″ z ) is used to characterize e″ z The sum of the real values ​​of the vectors in each dimension. w3 in e″ z If there are two or more dimensions with values ​​greater than 0.5, they are equal to the preset third weight factor, otherwise w3 = 0. The preset third weight factor is greater than 0.

[0073] Thus, by setting w2·r⊙I l The optimized target feature vector can generate a larger value in the CLS dimension of the model to be tested. By setting w3·sum(e″ z ) can reduce the trigger word vector x z There is only one dimension with a value greater than 0.5.

[0074] In the second embodiment, in the presence of a normal model that has not been attacked by the backdoor, the loss function LOSS MSE =w1·L ce (y z ,l)+w2·r⊙I l +w3·sum(x z )+w4·Lce (y′ z ,l0). Among them, y′ z is a normal model for random sampling of x z The predicted label of L ce (y′ z ,l0) is used to represent the predicted label y′ z The cross entropy loss L between the non-target label l0 ce This ensures that when the loss function converges to a minimum, the obtained trigger will not cause the random normal model to produce the same misclassification.

[0075] Optionally, obtaining a number of initial text vectors corresponding to the initial backdoor feature vector includes: selecting any one of a plurality of preset candidate word sets as an initial candidate word set corresponding to the initial backdoor feature vector. Adding the initial backdoor feature vector to a number of text vectors included in the initial candidate word set to obtain a number of initial text vectors corresponding to the initial backdoor feature vector.

[0076] Optionally, after obtaining the backdoor feature vector corresponding to the initial backdoor feature vector, the method further includes: triggering the model to be detected to perform a regularization operation at intervals of a preset time period.

[0077] Combination Figure 4 As shown, the embodiment of the present disclosure provides a method for detecting a backdoor attack, comprising:

[0078] Step S401, obtain the CLS feature dimension vector of the target label, and obtain the loss function according to the trigger input vector of the model to be detected and the CLS feature dimension vector.

[0079] Step S402, performing a backdoor feature vector determination operation according to a preset initial backdoor feature vector and a loss function until the loss function converges to a minimum value, and determining the backdoor feature vector corresponding to the initial backdoor feature vector as the target feature vector; wherein the backdoor feature vector operation includes: obtaining a number of initial text vectors corresponding to the initial backdoor feature vector; inputting each initial text vector into a model to be detected for iterative training, and obtaining initial probability values ​​corresponding to each initial text vector; performing calculations using each initial probability value according to the loss function until the number of iterations is equal to a preset threshold, and obtaining a backdoor feature vector corresponding to the initial backdoor feature vector, triggering the model to be detected to perform a regularization operation at intervals of a preset time period, and replacing the initial backdoor feature vector with the backdoor feature vector corresponding to the initial backdoor feature vector.

[0080] Step S403 , sorting the multiple text vectors included in the target feature vector in descending order according to the initial probability values, and determining a preset number of text vectors as candidate text vectors.

[0081] Step S404, obtaining candidate text sequences corresponding to each candidate text vector.

[0082] Step S405 , input each candidate text sequence into the model to be detected, and obtain a first probability value corresponding to each candidate text sequence, wherein the first probability value is used to characterize the probability that the candidate text sequence includes the target label.

[0083] Step S406: When there is a first probability value greater than a preset probability value, it is determined that the model to be detected has been attacked by a backdoor.

[0084] The method for detecting backdoor attacks provided by the embodiment of the present disclosure is adopted. By obtaining a loss function according to the trigger input vector of the model to be detected, and then performing a backdoor feature vector determination operation according to the preset initial backdoor feature vector and the loss function, it is possible to find the possible backdoor feature vector corresponding to the trigger input vector, that is, the target feature vector, and then determine the text vector with a larger probability value in the target feature vector as the candidate text vector, and obtain the candidate text sequence corresponding to each candidate text vector, and then input each candidate text sequence into the model to be detected to obtain the probability that the model to be detected is triggered by the candidate text vector, thereby outputting the target label. Then, in the case of a first probability value greater than the preset probability value, it is determined that the model to be detected has been attacked by the backdoor. In this way, it is possible to more accurately detect whether the model to be detected has been attacked by the backdoor.

[0085] Optionally, after obtaining the backdoor feature vector corresponding to the initial backdoor feature vector, it also includes: setting the number of delay optimization rounds at intervals to trigger the model to be detected to perform a regularization operation. Among them, one round of optimization is used to characterize the backdoor feature vector determination operation using a set number of initial candidate word sets. In this way, it can be ensured that both the word vector and the term vector contain distributed real values, and that the sum of the dimensional real values ​​of the trigger input vector x is 1.

[0086] Combination Figure 5 As shown, the embodiment of the present disclosure provides a method for detecting a backdoor attack, comprising:

[0087] Step S501, obtain the CLS feature dimension vector of the target label, and obtain the loss function according to the trigger input vector of the model to be detected and the CLS feature dimension vector.

[0088] Step S502, performing a backdoor feature vector determination operation according to a preset initial backdoor feature vector and a loss function until the loss function converges to a minimum value, and determining the backdoor feature vector corresponding to the initial backdoor feature vector as the target feature vector; wherein the backdoor feature vector operation includes: obtaining a number of initial text vectors corresponding to the initial backdoor feature vector; inputting each initial text vector into a model to be detected for iterative training, and obtaining initial probability values ​​corresponding to each initial text vector; performing calculations using each initial probability value according to the loss function until the number of iterations is equal to a preset threshold, and obtaining a backdoor feature vector corresponding to the initial backdoor feature vector, setting an interval to delay the number of optimization rounds to trigger the model to be detected to perform a regularization operation, and replacing the initial backdoor feature vector with the backdoor feature vector corresponding to the initial backdoor feature vector.

[0089] Step S503 , sorting the multiple text vectors included in the target feature vector in descending order according to the initial probability values, and determining a preset number of text vectors as candidate text vectors.

[0090] Step S504, obtaining candidate text sequences corresponding to each candidate text vector.

[0091] Step S505: input each candidate text sequence into the model to be detected to obtain a first probability value corresponding to each candidate text sequence, where the first probability value is used to represent the probability that the candidate text sequence includes the target label.

[0092] Step S506: When there is a first probability value greater than a preset probability value, it is determined that the model to be detected has been attacked by a backdoor.

[0093] The method for detecting backdoor attacks provided by the embodiment of the present disclosure is adopted. By obtaining a loss function according to the trigger input vector of the model to be detected, and then performing a backdoor feature vector determination operation according to the preset initial backdoor feature vector and the loss function, it is possible to find the possible backdoor feature vector corresponding to the trigger input vector, that is, the target feature vector, and then determine the text vector with a larger probability value in the target feature vector as the candidate text vector, and obtain the candidate text sequence corresponding to each candidate text vector, and then input each candidate text sequence into the model to be detected to obtain the probability that the model to be detected is triggered by the candidate text vector, thereby outputting the target label. Then, in the case of a first probability value greater than the preset probability value, it is determined that the model to be detected has been attacked by the backdoor. In this way, it is possible to more accurately detect whether the model to be detected has been attacked by the backdoor.

[0094] In some embodiments, the number of delay optimization rounds is set to 30, and the number of candidate word sets is set to 20. The model to be detected is triggered to use gumbel softmax for regularization operation in every 30 optimization rounds, which is compared with the triggering model to be detected using gumbel softmax to perform regularization operation in each optimization round. The restrictions on the probability values ​​of the CLS feature dimensions are relaxed, so that the dimensions associated with the real trigger words can obtain larger probability values. Among them, the essence of the gumbel softmax method is to use a continuously differentiable function to simulate the result expression of the argmax function, so that it can complete the function of the argmax function without truncating the gradient return.

[0095] Optionally, after obtaining the first probability values ​​corresponding to each candidate text sequence, it also includes: in the absence of a first probability value greater than a preset probability value, obtaining the linear weight parameters corresponding to each candidate text vector. Obtain the CLS feature dimension vector of the target label, and calculate using the linear weight parameters and the CLS feature dimension vector of the target label according to the second preset algorithm to obtain a discriminative verification parameter. Determine whether the discriminative verification parameter is greater than a preset judgment threshold. When the discriminative verification parameter is greater than the preset judgment threshold, determine that the model to be detected has been attacked by a backdoor. And / or, when the discriminative verification parameter is less than or equal to the preset judgment threshold, determine that the model to be detected has not been attacked by a backdoor. In this way, by superimposing the discriminative verification parameter verification, a more accurate backdoor detection result can be obtained.

[0096] Combination Figure 6 As shown, the embodiment of the present disclosure provides a method for detecting a backdoor attack, comprising:

[0097] Step S601, obtain the CLS feature dimension vector of the target label, and obtain the loss function according to the trigger input vector of the model to be detected and the CLS feature dimension vector.

[0098] Step S602, performing a backdoor feature vector determination operation according to a preset initial backdoor feature vector and a loss function until the loss function converges to a minimum value, and determining the backdoor feature vector corresponding to the initial backdoor feature vector as the target feature vector; wherein the backdoor feature vector operation includes: obtaining a number of initial text vectors corresponding to the initial backdoor feature vector; inputting each initial text vector into a model to be detected for iterative training, and obtaining initial probability values ​​corresponding to each initial text vector; performing calculations using each initial probability value according to the loss function, until the number of iterations is equal to a preset threshold, and obtaining a backdoor feature vector corresponding to the initial backdoor feature vector, and performing a replacement operation on the initial backdoor feature vector using the backdoor feature vector corresponding to the initial backdoor feature vector.

[0099] Step S603 , sorting the multiple text vectors included in the target feature vector in descending order according to the initial probability values, and determining a preset number of text vectors as candidate text vectors.

[0100] Step S604, obtaining candidate text sequences corresponding to each candidate text vector.

[0101] Step S605: input each candidate text sequence into the model to be detected to obtain a first probability value corresponding to each candidate text sequence, where the first probability value is used to represent the probability that the candidate text sequence includes the target label.

[0102] Step S606, determining whether there is a first probability value greater than a preset probability value, if yes, executing step S607, otherwise, executing step S608.

[0103] Step S607: determine whether the model to be detected has been attacked by a backdoor, and then end.

[0104] Step S608, obtaining the linear weight parameters corresponding to each candidate text vector, and then executing step S609.

[0105] Step S609, obtaining the CLS feature dimension vector of the target tag, and performing calculations using the linear weight parameter and the CLS feature dimension vector of the target tag according to the second preset algorithm to obtain a discriminative verification parameter.

[0106] Step S610, determining whether the discriminative verification parameter is greater than a preset discriminative threshold, if so, executing step S607, otherwise, executing step S611.

[0107] Step S611, determining whether the model to be detected has not been attacked by a backdoor.

[0108] The method for detecting backdoor attacks provided by the embodiment of the present disclosure is adopted. By obtaining a loss function according to the trigger input vector of the model to be detected, and then performing a backdoor feature vector determination operation according to the preset initial backdoor feature vector and the loss function, it is possible to find the possible backdoor feature vector corresponding to the trigger input vector, that is, the target feature vector, and then determine the text vector with a larger probability value in the target feature vector as the candidate text vector, and obtain the candidate text sequence corresponding to each candidate text vector, and then input each candidate text sequence into the model to be detected to obtain the probability that the model to be detected is triggered by the candidate text vector, thereby outputting the target label. Then, in the case of a first probability value greater than the preset probability value, it is determined that the model to be detected has been attacked by the backdoor. In this way, it is possible to more accurately detect whether the model to be detected has been attacked by the backdoor.

[0109] Optionally, obtaining the linear weight parameters corresponding to each candidate text vector includes: adding the candidate text vector to a portion of the text sequence included in the first text sequence set, and determining the label of the text sequence including the candidate text vector as 1, and determining the label of the text sequence not including the candidate text vector as 0. Inputting the labeled text sequence into the model to be detected, triggering the CLS word embedding layer of the model to be detected to output the real-valued parameters corresponding to the text sequence; inputting each real-valued parameter into a preset linear classifier for training, and obtaining the linear weight parameters corresponding to each candidate text vector.

[0110] In some embodiments, the real-valued parameters are input into a preset linear classifier for training to obtain linear weight parameters corresponding to the candidate text vector, and the linear weight parameters corresponding to the candidate text vector are brought into the preset linear classifier to obtain a label linear classifier corresponding to the candidate text vector. The label linear classifier corresponding to the candidate text vector can determine whether the candidate text vector is included in the input text sequence.

[0111] Optionally, according to a second preset algorithm, a linear weight parameter and a CLS feature dimension vector of the target label are used for calculation to obtain a discriminative verification parameter, including: performing a dot product of the linear weight parameter and the CLS feature dimension vector of the target label to obtain a dot product vector, and adding the values ​​included in the dot product vector to obtain a discriminative verification parameter.

[0112] In some embodiments, the test data set includes 3256 models, half of which are normal models and the other half are models that have been attacked by backdoors. The model architecture includes three commonly used natural language processing model architectures: DistilBERT, BERT, and GPT. All models are trained in the IMDB data set, the Amazon data set, and the CoNLL-2003 data set. The IMDB data set contains 50,000 IMDB movie reviews, which are specifically used for sentiment classification. The Amazon data set is a data set for binary sentiment classification. It contains 25,000 sample data for training and 25,000 test data for testing. The CoNLL-2003 data set is a named entity data set, which contains data related to four entities, including people, locations, organizations, and others. Among them, the backdoor attack method includes embedding word-level triggers and / or phrase-level triggers into random positions of the training text sequence, but keeping the semantics of the training text sequence unchanged. The training text sequence is used to train a natural language processing model.

[0113] In some embodiments, TP (True Positive) is used to evaluate the case where the actual result is a backdoor model and the detection result is also a backdoor model. TN (True Negative) is used to evaluate the case where the actual result is a backdoor model and the detection result is a normal model. FP (False Positive) is used to evaluate the case where the actual result is a normal model and the detection result is a backdoor model. FN (False Negative) is used to evaluate the case where the actual result is a normal model and the detection result is a normal model. The detection accuracy Acc (Accuracy) is used to evaluate the overall detection accuracy, which is formally defined as Acc = (TP + TN) / (TP + TN + FP + FN), indicating the ratio of correctly detected data to the total detection data. The larger the value, the better the detection accuracy and the better the detection effect.

[0114] Further, in combination with what is described in Table 1, the method Proposed for detecting backdoor attacks provided by the embodiment of the present disclosure is used to perform backdoor detection on a test data set with DistilBERT as the model architecture, and the obtained TP, FP, FN, TN and Acc values ​​are 325, 27, 35, 199, and 0.894, respectively. The existing method GBDA for detecting backdoor attacks is used to perform backdoor detection on a test data set with DistilBERT as the model architecture, and the obtained TP, FP, FN, TN and Acc values ​​are 254, 8, 106, 138, and 0.677, respectively. The existing method T-miner for detecting backdoor attacks is used to perform backdoor detection on a test data set with DistilBERT as the model architecture, and the obtained TP, FP, FN, TN and Acc values ​​are 30, 4, 330, 222, and 0.430, respectively. It can be seen that when the method for detecting backdoor attacks provided by the embodiment of the present disclosure performs backdoor detection on the test data set with DistilBERT as the model architecture, the values ​​of TP, FP, FN, TN and Acc obtained are all higher than those of the existing methods for detecting backdoor attacks. Therefore, the method for detecting backdoor attacks provided by the embodiment of the present disclosure can more accurately detect backdoor attacks on the data set with DistilBERT as the model architecture than the methods GBDA and T-miner.

[0115]

[0116] Table 1

[0117] In some embodiments, as shown in Table 2, the method Proposed for detecting backdoor attacks provided by the embodiment of the present disclosure is used to perform backdoor detection on a test data set with BERT as the model architecture, and the obtained TP, FP, FN, TN and Acc values ​​are 188, 21, 23, 202, and 0.898, respectively. The existing method GBDA for detecting backdoor attacks is used to perform backdoor detection on a test data set with BERT as the model architecture, and the obtained TP, FP, FN, TN and Acc values ​​are 118, 28, 93, 195, and 0.721, respectively. The existing method T-miner for detecting backdoor attacks is used to perform backdoor detection on a test data set with BERT as the model architecture, and the obtained TP, FP, FN, TN and Acc values ​​are 41, 6, 170, 217, and 0.594, respectively. It can be seen that when the method for detecting backdoor attacks provided by the embodiment of the present disclosure performs backdoor detection on the test data set with BERT as the model architecture, the values ​​of TP, FP, FN, TN and Acc obtained are all higher than those of the existing method for detecting backdoor attacks. Therefore, the method for detecting backdoor attacks provided by the embodiment of the present disclosure can more accurately detect backdoor attacks on the data set with BERT as the model architecture than the methods GBDA and T-miner.

[0118]

[0119] Table 2

[0120] In some embodiments, in combination with Table 3, the method for detecting backdoor attacks provided by the embodiment of the present disclosure is used to perform backdoor detection on a test data set with GPT as the model architecture, and the obtained TP, FP, FN, TN and Acc values ​​are 224, 26, 33, 195, and 0.877, respectively. The existing method GBDA for detecting backdoor attacks is used to perform backdoor detection on a test data set with GPT as the model architecture, and the obtained TP, FP, FN, TN and Acc values ​​are 140, 27, 117, 194, and 0.700, respectively. The existing method T-miner for detecting backdoor attacks is used to perform backdoor detection on a test data set with GPT as the model architecture, and the obtained TP, FP, FN, TN and Acc values ​​are 34, 16, 223, 205, and 0.500, respectively. It can be seen that when the method for detecting backdoor attacks provided by the embodiment of the present disclosure performs backdoor detection on the test data set with GPT as the model architecture, the values ​​of TP, FP, FN, TN and Acc obtained are all higher than those of the existing methods GBDA and T-miner for detecting backdoor attacks. Therefore, the method for detecting backdoor attacks provided by the embodiment of the present disclosure can more accurately detect backdoor attacks on the data set with GPT as the model architecture than the methods GBDA and T-miner.

[0121]

[0122] Table 3

[0123] In some embodiments, as shown in Table 4, the time for backdoor attack detection on the test data set with DistilBERT, BERT, and GPT as the model architecture using the existing method GBDA for detecting backdoor attacks is 281, 480, and 415, respectively. The time for backdoor attack detection on the test data set with DistilBERT, BERT, and GPT as the model architecture using the existing method T-miner for detecting backdoor attacks is 2829, 3272, and 3341, respectively. The time for backdoor attack detection on the test data set with DistilBERT, BERT, and GPT as the model architecture using the method Proposed for detecting backdoor attacks provided by the example of the present disclosure is 279, 353, and 513, respectively. It can be seen that the backdoor attack detection time of the method Proposed for detecting backdoor attacks provided by the embodiment of the present disclosure on the test data set with DistilBERT, BERT, and GPT as the model architecture is shorter than that of the existing methods GBDA and T-miner for detecting backdoor attacks. Therefore, the method for detecting backdoor attacks provided by the embodiments of the present disclosure can detect whether a model is attacked by a backdoor more quickly than the prior art.

[0124]

[0125]

[0126] Table 4

[0127] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0128] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods for each specific application to implement the described functions, but such implementations should not be considered to exceed the scope of the embodiments of the present disclosure. The technicians may clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above may refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0129] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0130] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for detecting backdoor attacks, characterized in that: include: Obtain a CLS feature dimension vector of the target label, and obtain a loss function according to the trigger input vector of the model to be detected and the CLS feature dimension vector of the target label; A backdoor feature vector determination operation is performed according to a preset initial backdoor feature vector and the loss function until the loss function converges to a minimum value, and the backdoor feature vector corresponding to the initial backdoor feature vector is determined as a target feature vector; wherein the backdoor feature vector operation includes: obtaining a number of initial text vectors corresponding to the initial backdoor feature vector; inputting each of the initial text vectors into a model to be detected for iterative training, and obtaining initial probability values ​​corresponding to each of the initial text vectors; performing calculations using each of the initial probability values ​​according to the loss function until the number of iterations is equal to a preset threshold, and obtaining a backdoor feature vector corresponding to the initial backdoor feature vector, and performing a replacement operation on the initial backdoor feature vector using the backdoor feature vector corresponding to the initial backdoor feature vector; Sorting the multiple text vectors included in the target feature vector in descending order according to the initial probability values, and determining a preset number of text vectors as candidate text vectors; Obtaining candidate text sequences corresponding to each of the candidate text vectors; the candidate text sequence is a text sequence including the corresponding candidate text vectors; Input each of the candidate text sequences into the model to be detected, and obtain a first probability value corresponding to each of the candidate text sequences, wherein the first probability value is used to represent the probability that the candidate text sequence includes the target label; In the case that there is a first probability value greater than a preset probability value, it is determined that the model to be detected has been attacked by a backdoor.

2. The method according to claim 1, characterized in that Get the CLS feature dimension vector of the target label, including: Randomly sampling from a preset first text sequence set to obtain a number of first sampling data; Obtaining a first candidate CLS feature real-valued vector of each of the first sampling data; Determine a real value variation range of a target CLS feature real value vector of the model to be detected according to each of the first candidate CLS feature real value vectors; Randomly sampling from a preset second text sequence set to obtain a number of second sampling data; Modify the second candidate CLS feature real-valued vector of the second sampled data a preset number of times according to the real-value variation range to obtain modified second sampled data; Inputting each of the modified second sampling data into the model to be detected to obtain a difference value corresponding to each of the modified second sampling data; The CLS feature dimension vector of the target label is obtained according to each of the difference values.

3. The method according to claim 2, characterized in that Obtaining a first candidate CLS feature real-valued vector of each of the first sampling data, including: Performing a differentiable coding transformation on each of the first sampling data to obtain a first data coding vector corresponding to each of the first sampling data; Obtain a feature representation model from the model to be detected; Inputting each of the first data encoding vectors into the feature representation model to obtain a feature representation vector corresponding to each of the first sampling data; A first candidate CLS feature real-valued vector of each of the first sampling data is obtained from each of the feature representation vectors.

4. The method according to claim 2, characterized in that: Determining a real value variation range of a target CLS feature real value vector of the model to be detected according to each of the first candidate CLS feature real value vectors includes: The minimum value of the Ith dimension in each of the first candidate CLS feature real-valued vectors is determined as the minimum value of the Ith dimension of the target CLS feature real-valued vector, and the maximum value of the Ith dimension in each of the first candidate CLS feature real-valued vectors is determined as the maximum value of the Ith dimension of the target CLS feature real-valued vector; I is a positive integer, and I is less than the maximum dimension value in the candidate CLS feature real-valued vector, and I is greater than 0; The variation range of the first dimension of the target CLS feature real-valued vector is determined as the minimum value of the first dimension of the target CLS feature real-valued vector to the maximum value of the first dimension of the target CLS feature real-valued vector.

5. The method according to claim 2, characterized in that: Modifying the second candidate CLS feature real-valued vector of the second sampled data a preset number of times according to the real-value variation range to obtain the modified second sampled data includes: According to the variation range of the first dimension of the target CLS feature real-valued vector, the vector value of the first dimension of each of the second candidate CLS feature real-valued vectors is modified a preset number of times to obtain a number of first-dimensional modified sampling data corresponding to each of the second sampling data.

6. The method according to claim 5, characterized in that Inputting each of the modified second sampled data into the model to be detected to obtain a difference value corresponding to each of the modified second sampled data, including: Input each of the first dimension modified sampling data into the model to be detected to obtain a third probability value corresponding to each of the first dimension modified sampling data, and the third probability value is used to characterize the probability that the first dimension modified sampling data includes the target label.

7. The method according to claim 6, characterized in that Obtaining the CLS feature dimension vector of the target label according to each of the difference values ​​includes: Determine the minimum third probability value among the third probability values ​​corresponding to the first dimension modified sampling data as the first dimension difference minimum value corresponding to the second sampling data; determine the maximum third probability value among the third probability values ​​corresponding to the first dimension modified sampling data as the first dimension difference maximum value corresponding to the second sampling data; According to the third preset algorithm, the vector value of the Ith dimension of the CLS feature dimension vector of the target label is obtained by calculating the minimum difference value of the Ith dimension corresponding to each of the second sampling data and the maximum difference value of the Ith dimension corresponding to each of the second sampling data.

8. The method according to claim 1, characterized in that The loss function is obtained according to the trigger input vector of the model to be tested, including: Obtaining a numerical restriction function according to a preset initial backdoor feature vector; Using the numerical limit function to limit the trigger input vector to obtain a limited input vector; Performing a differentiable coding transformation on the restricted input vector to obtain a second data coding vector; A loss function is obtained according to the second data encoding vector.

9. The method according to claim 1, characterized in that: After obtaining the first probability values ​​corresponding to the candidate text sequences respectively, the method further includes: In the absence of a first probability value greater than the preset probability value, obtaining linear weight parameters corresponding to each of the candidate text vectors; Calculate using the linear weight parameter and the CLS feature dimension vector of the target label according to a second preset algorithm to obtain a discriminative verification parameter; determining whether the discriminative verification parameter is greater than a preset discriminative threshold; In the case where the discriminative verification parameter is greater than a preset judgment threshold, determining that the model to be detected has been attacked by a backdoor; and / or, When the discriminative verification parameter is less than or equal to a preset discriminative threshold, it is determined that the model to be detected has not been attacked by a backdoor.

10. The method according to claim 9, characterized in that Obtaining the linear weight parameters corresponding to each of the candidate text vectors, including: Add the candidate text vector to a portion of the text sequences included in the first text sequence set, and determine the label of the text sequence including the candidate text vector as 1, and determine the label of the text sequence not including the candidate text vector as 0; Input the labeled text sequence into the model to be detected, triggering the CLS word embedding layer of the model to be detected to output the real-valued parameters corresponding to the text sequence; input each of the real-valued parameters into a preset linear classifier for training, and obtain the linear weight parameters corresponding to each of the candidate text vectors.

Images