A defense method against adversarial sample attack based on integrated reconstruction mechanism

Abstract

The application provides a defense method against an adversarial sample attack based on an integrated reconstruction mechanism, and is used for language model defense against an adversarial sample attack, and comprises the following steps: step 1, in the training process of the language model, three predetermined loss functions are simultaneously added to keep the reconstruction ability of the language model for a noisy sample; step 2, different noises are injected into the input sample of the language model, a plurality of different reconstructed samples are obtained through reconstruction, integrated inference is performed on the reconstructed samples to complete a sample reconstruction, and a scoring of a model confidence value obtained by averaging is obtained, and the adversarial sample attack is prevented through multiple sample reconstructions, wherein the loss function comprises a fine-tuning loss function of a downstream task of the model, a MASK mask prediction loss adopted in pre-training, and a loss based on gradient back propagation for reconstructing original words.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of deep learning and natural language processing, and particularly relates to a defense method against adversarial sample attacks based on an integrated reconstruction mechanism. BACKGROUND

[0002] Adversarial sample attacks can successfully mislead neural networks, causing the neural networks to make incorrect predictions. These methods usually use gradient-based perturbation generation methods to generate perturbations in the opposite direction of the loss function to attack the model. In the field of natural language, due to the discrete nature of the gradient in the text, the model cannot directly use the gradient to generate perturbations. Usually, people use a word replacement-based method to generate adversarial samples that can maximize semantic assurance while causing the neural network to make incorrect predictions. Therefore, the widespread application of neural networks cannot be separated from effective methods to defend against these adversarial sample attacks.

[0003] In the field of natural language processing, the generation of adversarial samples usually focuses on semi-black-box scoring models, that is, the attacker can perceive the confidence of the model output, and select appropriate adversarial samples by observing the change in the confidence of the generated adversarial samples. Researchers use search algorithms such as greedy algorithms and genetic algorithms to find appropriate positions and words to replace, add or delete to generate appropriate adversarial samples. In more recent work, people use pre-trained language models to generate appropriate perturbation samples.

[0004] Researchers have relatively less work on defending against these adversarial samples: people use gradient-based adversarial training to find virtual adversarial samples similar in word embedding layer space to simulate real adversarial samples for corresponding training to achieve defense. Another idea of the defense method is to construct a convex hull space by obtaining a set of word replacement of adversarial samples in advance to satisfy the perturbation in this space can be compatible with the model and correctly predicted.

[0005] However, the effect of the virtual gradient-based method is relatively not excellent, and the application scene of this kind of method of obtaining the perturbation word set in advance is more limited in real application. SUMMARY

[0006] The application is made to solve the above problems, and aims to provide a defense method against adversarial sample attacks based on an integrated reconstruction mechanism.

[0007] The application provides a defense method against adversarial sample attack based on an integrated reconstruction mechanism, which is used for language model defense against adversarial sample attack, and has the following characteristics: step 1, in the training process of the language model, three predetermined loss functions are simultaneously added to maintain the reconstruction ability of the language model for the noisy sample; step 2, different noises are injected into the input sample of the language model, a plurality of different reconstruction samples are obtained through reconstruction, integrated inference is performed on the reconstruction samples, a scoring of model confidence is obtained by averaging, and the plurality of reconstruction samples obtained through reconstruction are used to defend against adversarial sample attack, wherein the loss function includes a fine-tuning loss function of a downstream task of the model, a MASK mask prediction loss used in pre-training, and a loss of reconstruction of original words based on gradient back propagation.

[0008] In the defense method against adversarial sample attack based on the integrated reconstruction mechanism provided by the application, the fine-tuning loss function of the downstream task of the model can have the following formula:

[0009]

[0010] In formula (1), θ is a model parameter, L is a cross-entropy loss function, F is a model classifier, x is a reconstructed input, and y is an input label. c

[0011] In the defense method against adversarial sample attack based on the integrated reconstruction mechanism provided by the application, the fine-tuning loss function of the downstream task of the model can have the following formula:

[0012]

[0013] In formula (2), θ is a model parameter, L is a cross-entropy loss function, F is a model classifier, x is a reconstructed input, and y is an input label. m

[0014] In the defense method against adversarial sample attack based on the integrated reconstruction mechanism provided by the application, the loss of reconstruction of original words based on gradient back propagation can use the loss generated when the language model is fine-tuned for the downstream task and the language model is predicted, return the loss to the word embedding layer through gradient back propagation, construct a virtual sequence representation by directly accumulating the gradient on the vector representation of the word embedding layer, and make the model re-predict the original sequence, and the calculation formula is as follows:

[0015]

[0016] In formula (3), θ is a model parameter, L is a cross-entropy loss function, F is a model classifier, x is a reconstructed input, and y is an input label. m ​​​​for model language model classifier, for input before reconstruction, delta is model gradient, ||g δ || F for model Frobenius norm.

[0017] Effects of the invention

[0018] According to the defense method for the attack of the adversarial sample based on the integrated reconstruction mechanism, because three loss functions are simultaneously added to the training process of the language model, the model can realize the prediction ability of the original language model while fine-tuning the downstream task of the model, so that the model has the ability to reconstruct the input noisy sample, and can alleviate the attack effect of the adversarial sample through multiple reconstruction of the noisy sample; and the integrated mechanism is used for multiple sample reconstruction during inference, which can avoid the attack effect caused by individual disturbance to the model. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a flowchart of the defense method for the attack of the adversarial sample based on the integrated reconstruction mechanism in an embodiment of the present application;

[0020] Figure 2 is a reconstruction schematic diagram of the noisy sample by the trained language model in an embodiment of the present application;

[0021] Figure 3 is a flowchart of the process during integrated inference in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the technical means and effects of the present application easy to understand, the present application is specifically described below in combination with embodiments and drawings.

[0023] <EMBODIMENT>

[0024] Figure 1 is a flowchart of the defense method for the attack of the adversarial sample based on the integrated reconstruction mechanism in an embodiment of the present application.

[0025] As Figure 1 shown, the defense method for the attack of the adversarial sample based on the integrated reconstruction mechanism in the embodiment is used for language model defense against adversarial sample attack, which includes the following steps:

[0026] Step 1, in the training process of the language model, three predetermined loss functions are simultaneously added to maintain the reconstruction ability of the language model for the noisy sample.

[0027] The loss function includes the fine-tuning loss function of the downstream task of the model, the MASK mask prediction loss used in pre-training, and the loss of reconstructing the original word based on gradient backpropagation.

[0028] The formula of the fine-tuning loss function of the downstream task of the model is as follows:

[0029]

[0030] In formula (1), theta is a model parameter, L is a cross-entropy loss function, F c is a model classifier, is a reconstructed input, and y is an input label.

[0031] The formula of the MASK mask prediction loss used in pre-training is as follows:

[0032]

[0033] In formula (2), theta is a model parameter, L is a cross-entropy loss function, F m is a model classifier, is a reconstructed input, and y is an input label.

[0034] The loss of reconstructing the original word based on gradient backpropagation uses the loss generated when the language model is used to fine-tune the downstream task and predict the language model, and returns the loss to the word embedding layer through gradient backpropagation. A virtual sequence representation is constructed by directly accumulating the gradient on the vector representation of the word embedding layer, and the model is used to re-predict the original sequence. The calculation formula is as follows:

[0035]

[0036] In formula (3), theta is a model parameter, L is a cross-entropy loss function, F m is a model language model classifier, is an input before reconstruction, delta is a model gradient, and ||g δ || F is the Frobenius norm of the model.

[0037] In this embodiment, the neural network built by the pre-trained language model usually needs to be fine-tuned to perform specific functions of the downstream task, such as text classification. Therefore, the pre-trained language model no longer has effective prediction of missing semantics, which makes the fine-tuned model pay more attention to the prediction of the downstream task, and such training also makes the model more susceptible to adversarial samples.

[0038] Therefore, when training the language model with reconstruction capability, the method based on virtual adversarial training is adopted, that is, three loss functions are added in the training process to make the model maintain the reconstruction capability of the noisy sample, so that the tolerance of the model to the input sample can be greatly enhanced.

[0039] Step 2, injecting different noises into the input sample of the language model, reconstructing to obtain multiple different reconstructed samples, and performing ensemble inference on the reconstructed samples to obtain an average model confidence score.

[0040] Figure 2 is a reconstruction diagram of the language model trained in the embodiment of the application on a noisy sample.

[0041] As Figure 2 shown, by replacing "highly" in the original sample (Original) with "inordinately" to generate an adversarial sample (Adversary), the ordinary defense model will produce an incorrect prediction result, while the language model trained by the defense method against adversarial sample attacks based on the integrated reconstruction mechanism of the application can perform ensemble prediction (ensemble prediction) after multiple text reconstructions and obtain a correct prediction result, effectively defending against adversarial sample attacks.

[0042] Figure 3 is a flowchart of ensemble inference in the embodiment of the application.

[0043] As Figure 3 shown, different noises are injected into the input sample of the language model, multiple different reconstructed samples are reconstructed, and each reconstructed sample is predicted to obtain a prediction result, and then the prediction results are integrated to obtain an average model confidence score, thereby generating a comprehensive prediction result.

[0044] Effects of the embodiment

[0045] According to the defense method against adversarial sample attacks based on the integrated reconstruction mechanism, because three loss functions are added to the training process of the language model at the same time, the model can realize the prediction ability of the original language model while fine-tuning the downstream task, so that the model has the ability to reconstruct the input noisy sample, and can alleviate the attack effect of the adversarial sample by multiple reconstructions of the noisy sample; and the integrated mechanism is used for multiple sample reconstruction during inference, which can avoid the attack effect caused by individual perturbation to the model.

[0046] The above embodiments are preferred cases of the application and do not limit the protection scope of the application.

Claims

1. A defense method based on an integrated reconstruction mechanism against an adversarial sample attack, used for language model defense against adversarial sample attack, characterized in that, The method comprises the following steps: Step 1, in the training process of the language model, three predetermined loss functions are added at the same time to maintain the reconstruction ability of the language model for noisy samples; Step 2, different noises are injected into the input samples of the language model, a plurality of different reconstruction samples are obtained by reconstruction, integrated reasoning is performed on the reconstruction samples, a scoring of a model confidence average value is obtained, and a plurality of reconstruction samples are obtained by reconstruction to defend against adversarial sample attacks, Wherein, the loss function comprises a fine-tuning loss function of a downstream task of the model, a MASK mask prediction loss adopted in pre-training, and a loss of reconstructing original words based on gradient backpropagation, The formula of the fine-tuning loss function of the downstream task of the model is as follows: In formula (1), θ is a model parameter, L is a cross-entropy loss function, F c is a model classifier, is a reconstructed input, and y is an input label. The formula of the MASK mask prediction loss adopted in pre-training is as follows: In formula (2), θ is a model parameter, L is a cross-entropy loss function, F m is a model classifier, is a reconstructed input, and y is an input label. The loss of reconstructing original words based on gradient backpropagation utilizes the loss generated by the language model during downstream task fine-tuning and language model prediction, returns the loss to the word embedding layer through gradient backpropagation, constructs a virtual sequence representation by directly accumulating the gradient on the vector representation of the word embedding layer, and makes the model re-predict the original sequence, and the calculation formula is as follows: In Equation (3), θ is a model parameter, L is a cross-entropy loss function, F m is a model language model classifier, is an input before reconstruction, δ is a model gradient, ||g δ || F is a Frobenius norm on the model.

Images