An attention mechanism-based adversarial text defense method and system

Abstract

The application discloses an attention mechanism-based adversarial text defense method and system, which comprises the following steps: inputting to-be-recognized text into a natural language processing model with an encoder-decoder as a basic structure; calculating the importance score of each word in the text by using a word scoring function; taking the reciprocal of the importance score to form a reconstruction score vector; calculating the weight of each hidden layer vector according to an attention formula to obtain an attention weight vector; balancing the reconstruction score vector and the attention weight vector by using the way of multiplying the reconstruction score vector by a hyperparameter, multiplying corresponding elements in the reconstruction score vector and the attention weight vector one by one to obtain a final reconstruction attention vector; and multiplying the reconstruction attention vector by a hidden layer feature vector to obtain a reconstructed semantic code, and obtaining output after decoding. The application has good generalization performance, does not need to retrain a model when coping with new adversarial attacks, and has a certain effect on character-level adversarial attacks and word-level adversarial attacks.

Description

Technical Field

[0001] This invention relates to the fields of machine learning and natural language processing, and in particular to an adversarial text defense method and system based on an attention mechanism. Background Technology

[0002] With the rapid development of computer technology, the generation of big data from the internet, and improvements in neural network training methods, artificial intelligence (AI) has experienced rapid growth. Machine learning, a branch of AI, studies how to use computers to simulate or implement human learning activities. It is one of the most intelligent and cutting-edge research areas within AI. Since the 1980s, machine learning, as a means to achieve AI, has attracted widespread interest in the AI ​​community. Especially in the last decade or so, research in machine learning has developed rapidly, becoming one of the important topics in AI. Machine learning is not only applied in knowledge-based systems but also widely used in many other fields such as natural language understanding, non-monotonic reasoning, machine vision, and pattern recognition.

[0003] Deep neural networks (DNNs) have demonstrated powerful capabilities in solving major problems across various fields, such as computer vision, audio, and natural language processing. Due to their immense success, DNN-based systems are widely deployed in the physical world. However, DNN models are vulnerable to carefully crafted inputs with imperceptible perturbations. These modified inputs, known as adversarial examples, can pose potential security threats to DNN-based systems. Therefore, research into defense methods against adversarial examples is urgently needed.

[0004] Currently, research on adversarial examples and defenses in the image domain is quite extensive, but research on text, especially adversarial text defenses, is relatively limited. Existing methods for defending against adversarial attacks in text mainly include spell checking and adversarial training.

[0005] Spell checking is a special detection method in natural language processing, while adversarial training is a common defense method used in images, text, and audio. For example, Chinese patent document CN111046673A discloses an adversarial generative network and its training method for defending against malicious text samples. It utilizes a generator and a discriminator within the adversarial generative network framework for both malicious sample defense and generation. The generator consists of an autoencoder that maps discrete text data to a continuous high-dimensional hidden space, allowing the generator to generate malicious text using the hidden vectors. The discriminator, or discrimination model, is used to identify the data. The malicious text generated by the generator is labeled with real tags and simultaneously input into the discriminator along with real samples for training.

[0006] While these defense methods have achieved good results in their respective works, they also have some limitations. For example, spell checking works well in detecting character-level adversarial attacks, but its effectiveness is very limited in detecting word-level and sentence-level adversarial attacks. This limitation is due to the principle of the detection method itself. Adversarial training, on the other hand, involves training the model with adversarial samples and normal samples together, allowing the model to adapt to these adversarial samples. Therefore, this defense method has poor generalization performance and may not work properly when facing new attack methods. Summary of the Invention

[0007] This invention provides an adversarial text defense method based on an attention mechanism, which achieves defense against adversarial text by introducing an attention mechanism.

[0008] The technical solution of the present invention is as follows:

[0009] An attention-based adversarial text defense method includes the following steps:

[0010] (1) Input the text to be recognized into a natural language processing model with encoder-decoder as the basic structure, use the word scoring function to calculate the importance score of each word in the text, take the reciprocal of the importance score of each word to form the reconstructed score vector of the text.

[0011] (2) Calculate the weights of each hidden layer vector according to the attention formula to obtain the attention weight vector;

[0012] (3) Use the method of multiplying the hyperparameter with the reconstructed score vector to balance the reconstructed score vector and the attention weight vector. Multiply the corresponding elements in the reconstructed score vector and the attention weight vector one by one to obtain the final reconstructed attention vector.

[0013] (4) The reconstructed semantic encoding vector is obtained by multiplying the reconstructed attention vector with the hidden layer feature vector, and the output is obtained after decoding.

[0014] Considering that most existing adversarial attack methods are based on modifying important words in the text, this invention's adversarial text defense method, targeting sequence models with an encoder-decoder structure built on a recurrent neural network (RNN), utilizes an attention mechanism to reduce the weight of these important words during training. This desensitizes the natural language processing model under the encoder-decoder framework to a certain extent, rendering the adversarial text ineffective, achieving correct classification, and thus achieving the purpose of defense.

[0015] Step (1) includes:

[0016] (1-1) Calculate word x according to the scoring function formula. i Temporal score S(x) in the text i ):

[0017] S(x i )=F(x1,x2,...,x i-1 x i )-F(x1, x2, ..., x i-1 )

[0018] Where, x i Let F be the i-th word in the sentence; F is the classifier.

[0019] (1-2) Calculate word x according to the tail-series scoring function formula. i Tail-end time score TS(x) in the text i ):

[0020] TS(x i )=F(x i x i+1 x i+2 , ..., x n )-F(x i+1 x i+2 , ..., x T )

[0021] Where, x i Let F be the i-th word in the sentence; F is the classifier; and T is the total number of words in the text.

[0022] (1-3) Calculate word x according to the comprehensive scoring function formula. i Importance score c i :

[0023] c i =S(x i )+α*TS(xi )

[0024] Where α is a hyperparameter, and its value ranges from (0, 1].

[0025] (1-4) Take the inverse of the importance score for each word to obtain the final text reconstruction score vector RS:

[0026]

[0027] The optimal value for α is 0.3.

[0028] Step (2) includes: calculating the encoder hidden layer state h at each time step. j Compared to the decoder hidden state s at the previous time step i-1 The correlation between them is determined, and after performing softmax normalization, the attention weight vector a of the decoder hidden layer at each time step is obtained. ij The calculation formula is as follows:

[0029]

[0030] Among them, e ij The hidden state h of the encoder at a certain moment represents the state of the encoder hidden layer. j Compared to the decoder hidden state s at the previous time step i-1 The degree of correlation between them; i and j are time sequence numbers, i and j ∈ T, and T is the total number of words in the text.

[0031] Preferred, e ij The calculation formula is as follows, obtained using a multilayer perceptron (MLP):

[0032] e ij =MLP(s) i-1 h j )

[0033] Wherein, the encoder hidden layer state h at time j j It is determined by the encoder hidden layer state h from the previous time step. j-1 and the input x at the current time j It is certain that the formula is:

[0034] h j =f(h) j-1 x j )

[0035] Where f is the encoder's mapping function.

[0036] Step (3) includes: calculating the reconstructed attention vector R = [r1, r2, ..., r] using the reconstructed rating vector and attention weight vector. T ];

[0037]

[0038] Where β is a hyperparameter, and its value ranges from (0, 1]. To reconstruct the j-th value in the rating vector RS.

[0039] In step (4), the semantic encoding vector L = [l1, l2, ..., l] is reconstructed by weighted summation. T ], l i The calculation formula is:

[0040]

[0041] The method of this invention is applicable to natural language processing models under the Encoder-Decoder framework, including but not limited to recurrent neural networks (RNNs), and is effective against character-level and word-level adversarial attacks.

[0042] The present invention also provides an attention-based adversarial text defense system, including a memory and a processor; the memory is used to store program instructions; the processor is used to call the program instructions to execute the attention-based adversarial text defense method.

[0043] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0044] This invention, by reconstructing the attention vector, enables the model to minimize its focus on traditionally important words while maintaining accurate predictions. This prevents adversarial attacks from significantly impacting the model's confidence output after modifying these important words, thus rendering the adversarial text ineffective and achieving the defensive objective. This invention exhibits good generalization performance, eliminating the need to retrain the model to address new adversarial attacks; it is effective against both character-level and word-level adversarial attacks. Attached Figure Description

[0045] Figure 1 This is a flowchart of an attention-based adversarial text defense method. Detailed Implementation

[0046] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the embodiments described below are intended to facilitate the understanding of the present invention and do not limit it in any way.

[0047] This embodiment discloses an adversarial text defense method based on an attention mechanism. First, an importance score is calculated for each word in the text using a word scoring function, and its reciprocal is taken to obtain a reconstructed score vector. Then, the weights of each hidden layer vector are calculated according to the attention formula to obtain an attention weight vector. Finally, the two vectors are balanced by multiplying hyperparameters with the reconstructed score vector, and corresponding elements in the two vectors are multiplied one by one to obtain the final reconstructed attention vector. The semantic encoding obtained by multiplying the reconstructed attention vector with the hidden layer feature vector ensures that the model, while accurately predicting, minimizes its sensitivity to modifications of important words, rendering the adversarial text ineffective and thus achieving the defense objective. Figure 1 As shown, the method consists of three parts: calculating the reconstruction score, calculating the attention weights, and calculating the reconstruction attention vector.

[0048] This embodiment specifically includes the following steps:

[0049] 1) Calculate the reconstruction score;

[0050] In this embodiment, the importance of a word is first determined by the difference in the classifier results after removing a word. The scoring function formula is as follows:

[0051] S(x i )=F(x1,x2,...,x i-1 x i )-F(x1, x2, ..., x i-1 )

[0052] Where, x i Let be the i-th word in the sentence; F is the classifier. The problem with the above scoring function is that it ignores the impact of deleted words on subsequent words. Therefore, a tail-time scoring function is needed, and its formula is as follows:

[0053] TS(x i )=F(x i x i+1 x i+2 , ..., x T )-F(x i+1 x i+2 , ..., x T )

[0054] Where T is the total number of words in the text.

[0055] At this point, we obtain the impact of removing the preceding and following words on the entire sentence. Finally, we synthesize the temporal score S(x) obtained above. i ) and tail time series score TS(x iWe use a hyperparameter α to balance the two scores, and finally obtain a comprehensive score function.

[0056] The formula for the comprehensive scoring function is as follows:

[0057] CS = S + α * TS

[0058] In this embodiment, the hyperparameter α is set to 0.3. The overall score CS is then obtained, which represents the x value of each word in the sentence. i Importance score c i ,Right now:

[0059] CS = [c1, c2, ..., c i c T ]

[0060] Finally, the reconstructed scoring function is calculated, and the formula for the reconstructed scoring function is as follows:

[0061]

[0062] in,

[0063] 2) Calculate attention weights;

[0064] In this embodiment, the weights are calculated using the attention formula. In a recurrent neural network (RNN), the hidden layer state h at the current time step... t It is determined by the hidden layer state h from the previous time step. t-1 and the input x at the current time t The formula is as follows:

[0065] h t =f(h) t-1 x t )

[0066] Current decoder hidden state s t From the hidden state s of the previous time step t-1 The decoder output y at the previous moment t-1 and semantic encoding vector l t The calculation is as follows:

[0067] s t =f(s) t-1 y t-1 , l t )

[0068] Get the current decoder hidden state s t Then the decoder output y at the current time can be obtained through calculation. t The calculation formula is as follows:

[0069] y t =g(y t-1 s t , l t )

[0070] Where g is the softmax function.

[0071] Next, we calculate the encoder hidden layer states h1 to h2 at each time step. T With decoder hidden layer state s t-1 The correlation between them is determined, and the weights 'a' of the decoder hidden layer vectors at each time step are obtained after performing softmax normalization. ij The calculation formula is as follows:

[0072]

[0073] Among them, e ij This represents the hidden state s of the decoder at the previous time step. i-1 With the encoder hidden layer vector h at the current time j The correlation between them is calculated using a multilayer perceptron (MLP), and the calculation formula is as follows:

[0074] e ij =MLP(s) i-1 h j )

[0075] 3) Calculate the reconstructed attention vector;

[0076] In this embodiment, the reconstruction score and attention weight obtained in steps 1) and 2) respectively are used to calculate the final reconstruction attention vector R = [r1, r2, ..., r T ],in,

[0077]

[0078] Where β is a hyperparameter; To reconstruct the j-th value in the score RS. In this embodiment, the hyperparameter β is set to 0.1.

[0079] After obtaining the reconstructed attention vector R, the semantic encoding vector L = [l1, l2, ..., l] can be further calculated by weighted summation. T ], l i The calculation formula is as follows:

[0080]

[0081] Next, decoding can proceed as normal, and then the next output is generated based on the region of interest. By reconstructing the attention vector, the model can minimize its focus on traditionally important words while ensuring accurate prediction. This prevents adversarial attacks from significantly affecting the model's confidence output after modifying these important words, rendering the adversarial text ineffective and achieving the defensive objective.

[0082] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An attention mechanism-based adversarial text defense method, characterized in that, The method comprises the following steps: (1) inputting to-be-identified text into a natural language processing model with an encoder-decoder as a basic structure, calculating an importance score of each word in the text by using a word scoring function, taking an inverse of the importance score of each word to form a reconstruction score vector of the text, comprising: (1-1) Calculate word according to scoring function formula Timing score in text : ； wherein, is the i-th word in the text; F is the classifier; (1-2) Calculating words according to the tail timing score function formula Tail timing score in text : ； wherein, is the i-th word in the text; F is the classifier; T is the total number of words in the text; (1-3) Calculate the importance score of the word according to the comprehensive score function formula :​ ； wherein, is a hyperparameter, with a value range of (0, 1]; (1-4) Inverse the importance score of each word to get the final reconstruction score vector of the text : ； (2) calculating a weight of each hidden layer vector according to an attention formula to obtain an attention weight vector; (3) using a manner of multiplying a hyperparameter and the reconstruction score vector to balance the reconstruction score vector and the attention weight vector, multiplying corresponding elements in the reconstruction score vector and the attention weight vector one by one to obtain a final reconstruction attention vector; (4) multiplying the reconstruction attention vector and a hidden layer feature vector to obtain a reconstruction semantic encoding vector, and obtaining an output after decoding.

2. The attention mechanism based adversarial text defense method according to claim 1, characterized in that, The value is 0.

3.

3. The attention mechanism based adversarial text defense method according to claim 1, characterized in that, Step (2) includes: calculating the correlation degree between the encoder hidden layer state at each time point and the decoder hidden layer state at the previous time point and performing a softmax normalization operation to obtain an attention weight vector of the decoder hidden layer at each time point , and the calculation formula is: ； wherein, represents the degree of correlation between the encoder hidden layer state at a certain time and the decoder hidden layer state at the previous time; i, j is the time index, i, j , T is the total number of words in the text.​ 4. The attention mechanism based adversarial text defense method according to claim 3, characterized in that, By multi-layer perceptron calculation, the calculation formula is: ； wherein, j encoder hidden layer state at a time instant is determined from the encoder hidden layer state at the previous time instant and the input at the current time instant is determined, and its formula is: ； wherein f is a mapping function of the encoder.

5. The attention mechanism based adversarial text defense method according to claim 4, characterized in that, Step (3) comprises: calculating a reconstruction attention vector using the reconstruction score vector and the attention weight vector ; ； wherein, is a hyperparameter, taking values in the range (0, 1]; is the value of the th value in the reconstruction score vector j th value in the reconstruction score vector 6. The attention mechanism-based adversarial text defense method according to claim 5, characterized in that, The value is 0.

1.

7. The attention mechanism-based adversarial text defense method of claim 5, wherein, In step (4), the reconstructed semantic encoding vector L is calculated by a weighted summation ], The calculation formula is: .

8. An attention mechanism based adversarial text defense system, characterized in that, The method comprises the following steps: (1) inputting to-be-identified text into a natural language processing model with an encoder-decoder as a basic structure, calculating an importance score of each word in the text by using a word scoring function, taking an inverse of the importance score of each word to form a reconstruction score vector of the text, comprising: (2) calculating a weight of each hidden layer vector according to an attention formula to obtain an attention weight vector; (3) using a manner of multiplying a hyperparameter and the reconstruction score vector to balance the reconstruction score vector and the attention weight vector, multiplying corresponding elements in the reconstruction score vector and the attention weight vector one by one to obtain a final reconstruction attention vector; (4) multiplying the reconstruction attention vector and a hidden layer feature vector to obtain a reconstruction semantic encoding vector, and obtaining an output after decoding. i, j i, j The method comprises the following steps: (1) inputting to-be-identified text into a natural language processing model with an encoder-decoder as a basic structure, calculating an importance score of each word in the text by using a word scoring function, taking an inverse of the importance score of each word to form a reconstruction score vector of the text, comprising: (2) calculating a weight of each hidden layer vector according to an attention formula to obtain an attention weight vector; (3) using a manner of multiplying a hyperparameter and the reconstruction score vector to balance the reconstruction score vector and the attention weight vector, multiplying corresponding elements in the reconstruction score vector and the attention weight vector one by one to obtain a final reconstruction attention vector; (4) multiplying the reconstruction attention vector and a hidden layer feature vector to obtain a reconstruction semantic encoding vector, and obtaining an output after decoding. i, j i, j The method comprises the following steps: (

Images