A Method of Preventing Text Stream Sequence Transformation Attack

Abstract

The invention realizes a method for preventing text flow sequence transformation attack through the method in the field of artificial intelligence. The method has three steps: learning the randomized sequence probability distribution based on the EM algorithm; generating the randomized sequence key; the training and deployment of the method needs to build an operating environment including pytorch and dependent libraries. The optimal sequence probability distribution for resisting attacks is learned through the EM algorithm, and finally the adversarial samples against flow sequence attacks cannot be accurately encoded in the attack sequence, achieving the effect of preventing text flow sequence attacks. The method of the invention has the best performance in resisting attacks, and also has the technical effect of weakening the attack strength of other text attacks.

Description

A method to prevent text flow order transformation attack technical field [0001] The present invention relates to the field of artificial intelligence, and in particular, to a method for preventing text flow order transformation attacks. Background technique [0002] Now natural language processing has been widely used in real life, such as machine translation, automatic summarization, question answering System and so on. The current mainstream natural language processing models such as Bert (Bidirectional Encoder Representation from Transformers) and its variants, etc., are improved based on the Transformer model, It mainly relies on its efficient self-attention mechanism and long-dependency alignment ability. However, the self-attention mechanism alone cannot capture the sequence The order information in the column, so it is necessary to add positional coding to fully characterize the sequence. But such a positional encoding is only It is the trigonometric funct...

AI Technical Summary

Problems solved by technology

However, the self-attention mechanism cannot capture the order information in the sequence alone, so it is necessary to add position encoding to fully represent the sequence

However, such position encoding is only a trigonometric function combined with order information, so it only needs to destroy the linearity of the trigonometric function in the position encoding, and it can effectively destroy the order information in the text to reduce the effectiveness of the model.

[0003] Therefore, the current mainstream natural language processing models based on transformer improvements need to use the linearity of trigonometric functions to obtain order information, but such encodings are subject to order attacks, and through research and experiments, we have found that Several methods for generating adversarial examples for sequential encoding. Such adversarial examples can effectively reduce the effectiveness of the model and are not easily detected by humans. The conclusion is that a means of preventing such attacks is currently needed to enhance the security of the model , which is of great significance to the security issues of the mainstream natural language processing model application

[0004] As mentioned above, the current mainstream natural language processing has added sequence coding that utilizes the linearity of trigonometric functions. Attacking sequence coding can easily reduce the accuracy of natural language processing models, but there is no complete targeting of flow sequence. How to defend against attacks

Images