Adversarial text defense method and system based on attention mechanism

Abstract

The invention discloses an adversarial text defense method and system based on an attention mechanism. The method comprises the following steps: inputting a to-be-recognized text into a natural language processing model which takes encoder-decoder as a basic structure, calculating an importance score of each word in the text through a word score function, and taking reciprocals of the importance scores to construct a reconstruction score vector; calculating a weight of each hidden layer vector according to an attention formula to obtain an attention weight vector; balancing the reconstruction score vector and the attention weight vector by using a mode of multiplying a hyper-parameter by the reconstruction score vector, and multiplying corresponding elements in the reconstruction score vector and the attention weight vector one by one to obtain a final reconstruction attention vector; and multiplying the reconstruction attention vector by a hidden layer feature vector to obtain a reconstruction semantic code, and performing decoding to obtain output. The invention is good in generalization performance, does not need to retrain a model when dealing with a new adversarial attack, and has a certain effect on character-level adversarial attacks and word-level adversarial attacks.

Description

Technical field [0001] The present invention relates to a machine learning, natural language processing, and more particularly, to a method and system for defense against text-based attentional mechanisms. Background technique [0002] With the rapid development of computer technology, the Internet has improved large data and neural network training methods, and artificial intelligence has developed rapidly. Machine learning is how a research in the field of artificial intelligence, or using a computer simulation to achieve scientific study of human activity, is the most intelligent artificial intelligence features, one of the most cutting-edge research. Since the 1980s, machine learning as a way to achieve artificial intelligence, artificial intelligence community has aroused widespread interest, especially in the last decade, research in machine learning has developed rapidly, it has become an important artificial intelligence one of the topics. Machine learning has been applie...

AI Technical Summary

Problems solved by technology

For example, spell checking is effective in detecting character-level adversarial attacks, but it is very limited in detecting word-level and sentence-level adversarial attacks. This limitation is caused by the principle of the detection method itself; while adversarial training is It is to participate in the training of the model with adversarial samples and normal samples, and let the model adapt to these adversarial samples. Therefore, the generalization performance of this defense method is poor, and it may not work properly in the face of new attack methods.

Images