Method for defending Chinese text adversarial attack based on soft prompt tuning model

Abstract

The invention discloses a method for defending Chinese text adversarial attacks based on a soft hint tuning model, which comprises the following steps of: 1) generating target domain data by using five Chinese-specific text attack strategies, and introducing a source domain and a target domain to simulate an original text and an adversarial text; 2) training an initial soft prompt tuning model based on a small amount of source domain data, and distributing a pseudo tag for a target domain instance; 3) dividing the target domain instance into n subsets, obtaining n tags of the target domain instance through multiple iterations, and analyzing syntactic features in combination with a graph attention model; and 4) selecting consistent labels in multiple iterations through a voting mechanism to serve as final training data for model training, and realizing classification prediction of the Chinese adversarial text. According to the method, an initial model is trained through a small amount of source domain data, real label information of a target domain is revealed, and accurate classification is realized through a voting mechanism and a soft prompt tuning model of graph attention enhanced Chinese grammar features.

Description

Technical Field

[0001] The present invention relates to the research fields of text adversarial and transfer learning, and in particular to a method for defending against adversarial attacks on Chinese text based on a soft prompt tuning model. Background Art

[0002] Adversarial attack is a technique that generates adversarial samples by applying subtle perturbations to input data. Its purpose is to mislead machine learning models, thereby revealing the model's vulnerabilities and providing research directions for enhancing the model's robustness and interpretability. Common defense methods mainly include adversarial training and knowledge distillation. Adversarial training introduces adversarial samples to train models to improve the model's immunity to perturbations; knowledge distillation optimizes the neural network structure to train the model to be more robust to adversarial perturbations, thereby reducing its vulnerability to attacks. However, due to the rich semantic hierarchy, high contextual dependence, and polysemy of Chinese, subtle text modifications may lead to significant semantic changes. Therefore, how to achieve efficient adversarial defense in Chinese scenarios remains an urgent problem to be solved.

[0003] Domain adaptation methods can be divided into two types: shallow learning and deep learning. Shallow learning methods usually rely on statistical model strategies and perform well in terms of accuracy and stability. However, in the scenario of cross-domain feature representation learning, the effect of shallow learning methods is still limited, which restricts their performance in domain adaptation. In contrast, deep learning methods have received widespread attention in domain adaptation research in recent years due to their powerful feature extraction capabilities. This type of method significantly improves the adaptability of the model by mapping the features of the source domain and the target domain to a shared new feature space. However, the optimization process of deep neural networks usually requires a large amount of labeled or unlabeled training data, and in practical applications, obtaining enough training samples often faces considerable difficulties. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the prior art and provide a method for defending against adversarial attacks on Chinese text based on a soft prompt tuning model. The source domain and the target domain are introduced to simulate the original text and the adversarial text respectively, thereby converting the text adversarial attack and defense tasks into domain adaptation tasks; text adversarial defense can be regarded as minimizing the deviation between domains, effectively reducing the dependence on a large amount of labeled data.

[0005] The object of the present invention is achieved by: a method for defending against adversarial attacks on Chinese text based on a soft prompt tuning model, comprising the following steps:

[0006] 1) Generate target domain data using five Chinese-specific text attack strategies, and introduce source and target domains to simulate original and adversarial texts;

[0007] 2) Train the initial soft hint tuning model based on a small amount of source domain data and assign pseudo labels to target domain instances;

[0008] 3) Divide the target domain instances into n subsets, each of which consists of pseudo labels and instances. Select one subset for training in each iteration, and use the other subsets for testing. Obtain n labels for the target domain instances through multiple iterations, and analyze syntactic features in combination with the graph attention model.

[0009] 4) Through the voting mechanism, the labels that are consistent in multiple iterations are selected as the final training data for model training to achieve classification prediction of Chinese adversarial text.

[0010] As a further limitation of the present invention, the step 1) specifically comprises:

[0011] Step 1.1) For the original text x = {x1, x2, ..., x n}Apply the greedy search strategy to each word x i Replace with unknown token UNK, generate new text x ′ ; Then use model F to measure the decrease in confidence. If the decrease in confidence exceeds 20%, it indicates that x i The prediction of the model is more important; the process is shown in formula (1), where y orig Represents the original label;

[0012]

[0013] stx ′ =replace(x,x i ,UNK)

[0014] Step 1.2) Sort the words in the input sentence by importance according to step 1.1); replace the words in descending order of importance until an adversarial text is generated that is sufficient to cause the model to make wrong predictions; the specific replacement strategies include the following five:

[0015] Open HowNet: This strategy uses the semantic similarity provided by Open HowNet to find the top k synonyms that are semantically closest to the target word for replacement;

[0016] Masked language model-based replacement: This strategy uses the pre-trained XLM-RoBERTa masked language model to predict alternative words for the target word and selects the top k words with the highest scores as candidate words;

[0017] Homophone replacement: This strategy looks for characters with the same pronunciation as the target word based on the pinyin of Chinese characters, and randomly selects the first k words from the homophone candidate set for replacement;

[0018] Homograph replacement: This strategy looks for Chinese characters that are visually similar to the target word based on the morphological structure of Chinese characters, and randomly selects the first k words from the lexical candidate set for replacement;

[0019] Combined substitution: This strategy combines the above four strategies by sequentially substituting Open HowNet, masked language model-based substitution, homophone substitution, and homograph substitution into the target word; if the substitution successfully changes the model’s prediction, the process stops; otherwise, the strategy continues with the next most important word until the model’s prediction is successfully perturbed;

[0020] After perturbation by five strategies, the original text and adversarial text are used as the source domain and target domain, respectively, thus converting the text adversarial defense task into a domain adaptation task.

[0021] As a further limitation of the present invention, the step 2) specifically comprises:

[0022] Step 2.1) Use the automatically generated template, i.e., the soft prompt template. The labeled sentence in the source domain is a sequence of words, represented as x = {x0, ..., x i ,...,x h}; In the embedding layer of PLM, each token is mapped into a pre-trained embedding space after disambiguation, where each token corresponds to a vector of fixed size; these embedding vectors encapsulate the semantic meaning of each token and represent its contextual understanding in the entire corpus. The labeled sentence is mapped to e(x) = {e(x0),...,e(x)} via PLMs e∈M. i ),...,e(x h )}, M refers to the Chinese-RoBERTa-WM-EXT model, and the embedding e of Chinese-RoBERTa-WM-EXT is introduced as PLMs M; the soft prompt T is then composed of the embedding sequence e(x), soft label h i And the mask embedding e(mask) is composed of:

[0023] T={h0,...,h i ,e(mask),h i+1 ,...,h n ,e(x)} (2)

[0024] Where i refers to the index of the soft marker, h i To represent soft tokens, soft hint templates replace natural language words in handcrafted templates;

[0025] Step 2.2) Four different strategies are used to build a label word mapper V t, in order to fully capture the various characteristics of the expansion words; the four different strategies include:

[0026] FastText Similarity: Calculates the semantic similarity between FastText vectors and the extended label words for each category;

[0027] Probabilistic prediction: Use the pre-trained language model to generate a probability distribution p(·|T[MASK]) for the word "[MASK]" in the vocabulary; select the top N words with the highest semantic relevance from this probability distribution a words; using contextual nuances in the sentence to predict the words that best fit the category;

[0028] Context information: Considering context information, the expanded vocabulary is required to include the surrounding vocabulary before and after the [MASK] tag; before entering BERT, [MASK] is replaced with these expanded vocabulary, and the top N words with the highest semantic relevance are selected through cross entropy loss calculation. a words;

[0029] Probability distribution optimization: The probability distribution is optimized by using the standard deviation to enhance the discrimination between label words and categories.

[0030] Step 2.3) By using the PromptForClassification function in the openprompt library, a prompt learning model PromptModel is constructed for the classification task; the PromptForClassification function accepts three parameters: the pre-trained language model plm, the template template, and the label word mapper verbalizer; the Bert model is used as the pre-trained language model, and the template prepared in step 2.1) and the label word mapper constructed in step 2.2) are assigned to the three parameters accordingly to complete the construction process of the soft prompt tuning model;

[0031] Step 2.4) uses a small portion of the source domain data according to steps 1) and 2.3) to train an initial soft-hint fine-tuning model, which is then used to assign pseudo labels to instances in the target domain.

[0032] As a further limitation of the present invention, the step 3) specifically comprises:

[0033] Step 3.1) Use the graph attention network to integrate the information of adjacent nodes in the syntactic graph and mine the syntactic structure of Chinese text. Given a Chinese sentence x = {x1,...,x i ,...,x n}, where x i Represents the i-th word; using the language technology platform, assign a part-of-speech tag P = {p1,...,pi ,...,p n}, where p i represents the part-of-speech tag of the i-th word; creates a directed graph G to represent the syntactic structure of Chinese text; each node in the directed graph is initialized with a randomly generated 150-dimensional feature vector to define its initial state; the syntactic structure feature f sem The calculation is as follows:

[0034] e ij =LeakyReLU(α T [Z i ‖Z j ]) (3)

[0035] l i =Elu(∑ j softmax(e ij )Z j W) (4)

[0036]

[0037] Among them, Z i represents the feature matrix of node i, α and W are learnable parameters, e ij is the attention weight between nodes i and j, l i represents the final node embedding obtained by applying the activation function to the weighted sum of adjacent node embeddings according to the attention weights, w i For each node feature l i The weight of the assignment;

[0038] Step 3.2) After the soft hint fine-tuning model initialized according to step 2.4) is trained in the source domain and tested in the target domain, each sample in the target domain is assigned a pseudo-label; an iterative soft hint fine-tuning model is introduced to improve the recognition of the true label from the predicted pseudo-label pool; all samples and their corresponding pseudo-labels in the target domain are initially divided into n segments, one of which is selected as the training dataset for the first round of hint fine-tuning model training, and the rest is used as the test data for label prediction; for each instance x∈X in the target domain, where X represents the set of all instances, the template is denoted as T(·), and the learning objective is expressed as:

[0039]

[0040] After the first data segment is used to train the initial soft prompt tuning model, pseudo labels are assigned to all other instances except the instances used in training; then, based on the first model, the second model is trained using the second segment in the n partitions through an iterative process; this cycle is repeated until each segment of the n partitions is trained once; during the iterative training, the syntactic structure feature analysis is performed in combination with the graph attention network constructed in step 3.1).

[0041] As a further limitation of the present invention, the step 4) specifically comprises:

[0042] Step 4.1) Through this iterative process of step 3.2), the soft hint tuning model generates n pseudo labels for each instance in the target domain; these pseudo labels are then screened through a voting mechanism to identify consistent labels and determine the final label set from the intersection of all votes; the final label set is used to train the final iterative model;

[0043] Step 4.2) After training through step 4.1), the sentences in the target domain are embedded and their label categories are divided into y∈Y. In the pre-trained language model, the label word mapper V t The probability of each label word being filled in [MASK] is expressed as P([MASK] = y|x p ), the text adversarial defense problem is regarded as a probability problem about label assignment, and the prediction probability formula is expressed as:

[0044] P(y∈Y|x)=P([MASK]=y|x p ) (7)

[0045] Step 4.3) Find V t After the predicted probability of each label word in , the predicted probability of each label word is mapped to the corresponding class label, and the average of these predicted probabilities is used for the final classification; the predicted probability of the final class label is expressed as:

[0046] P(y|x)=f(P M |v∈V t ) (8)

[0047] Step 4.4) In the target domain sentence, given the input x t If we calculate y t = The probability of {positive} is greater than y t ={negative}, it means the sentence belongs to the positive class.

[0048] The present invention adopts the above technical scheme, and compared with the prior art, the beneficial effects are as follows: the present invention combines the graph attention network with the soft prompt tuning model, and uses fusion features to solve the difficult problem of the Chinese text adversarial defense task; gives full play to the advantages of prompt tuning and effectively reduces the dependence on a large amount of labeled data; for the Chinese text adversarial defense task, a method for defending against Chinese text adversarial attacks based on a soft prompt tuning model is proposed, and after converting the text adversarial defense into a cross-domain classification task, the real label information is found in the target domain through an iterative soft prompt tuning method, and excellent performance is achieved; the present invention trains the initial model with a small amount of source domain data to reveal the real label information of the target domain, and enhances the soft prompt tuning model of Chinese grammatical features through a voting mechanism combined with graph attention to achieve accurate classification. The method of the present invention effectively defends against various text attacks and improves the robustness of Chinese text classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 The overall framework diagram of the present invention. DETAILED DESCRIPTION

[0050] like Figure 1 The method for defending against adversarial attacks on Chinese text based on a soft prompt tuning model shown in the figure includes the following steps:

[0051] 1) Generate target domain data using five Chinese-specific text attack strategies, and introduce source and target domains to simulate original and adversarial texts;

[0052] Step 1.1) First, the original text x = {x1, x2, ..., x n}Apply the greedy search strategy to each word x i Replace with unknown token UNK, generate new text x ′ ; Then use model F to measure the decrease in confidence. If the decrease in confidence exceeds 20%, it indicates that x i The prediction of the model is more important; the process is shown in formula (1), where y orig Represents the original label;

[0053]

[0054] stx ′ =replace(x,x i ,UNK)

[0055] Step 1.2) Sort the words in the input sentence by importance according to step 1.1); replace the words in descending order of importance until an adversarial text is generated that is sufficient to successfully deceive the model; the specific replacement strategies include the following five:

[0056] Open HowNet: This strategy uses the semantic similarity provided by Open HowNet to find the top k synonyms that are semantically closest to the target word for replacement; for example, the input sentence is "The product is very good, very satisfied!", after replacement it may become "The product is abnormally good, very satisfied!";

[0057] Replacement based on masked language model: This strategy uses the pre-trained XLM-RoBERTa masked language model to predict the replacement words of the target word, and selects the top k words with the highest scores as candidate words; for example, the input sentence is "The product is very good, very satisfied!", after replacement, it may become "The product is very good, very missing!";

[0058] Homophone replacement: This strategy looks for characters with the same pronunciation as the target word based on the pinyin of Chinese characters, and randomly extracts the first k words from the homophone candidate set for replacement; for example, the input sentence is "The product is very good, very satisfied!", after replacement it may become "The product is very good, very satisfied!";

[0059] Homograph replacement: This strategy looks for Chinese characters that are visually similar to the target word based on the morphological structure of Chinese characters, and randomly extracts the first k words from the lexical candidate set for replacement. For example, if the input sentence is "The product is very good, very satisfied!", after replacement, it may become "The product is very good, very satisfied!";

[0060] Combined substitution: This strategy combines the above four strategies together by sequentially substituting Open HowNet, masked language model-based substitution, homophone substitution, and homograph substitution into the target word; if the substitution successfully changes the model’s prediction, the process stops; otherwise, the strategy continues with the next most important word until the model’s prediction is successfully perturbed.

[0061] After perturbation by five strategies, the original text and adversarial text are used as the source domain and target domain, respectively, thus converting the text adversarial defense task into a domain adaptation task.

[0062] 2) Train the initial soft hint tuning model based on a small amount of source domain data and assign pseudo labels to target domain instances;

[0063] Step 2.1) Hard templates (i.e., handcrafted templates) in prompt fine-tuning have demonstrated fairly good results in various downstream tasks, but they rely on prior knowledge and are labor-intensive. To address this problem, an automatically generated template, namely a soft prompt template, is used; the labeled sentence in the source domain is a word sequence, represented as x = {x0,...,x i ,...,x h}; In the embedding layer of PLM, each token is mapped into a pre-trained embedding space after disambiguation, where each token corresponds to a vector of fixed size; these embedding vectors encapsulate the semantic meaning of each token and represent its contextual understanding in the entire corpus; in the experiment, the labeled sentence is mapped to e(x) = {e(x0),...,e(x i ),...,e(x h )}, M refers to the Chinese-RoBERTa-WM-EXT model, and the embedding e of Chinese-RoBERTa-WM-EXT is introduced as PLMs M; the soft prompt T is then composed of the embedding sequence e(x), soft label h i And the mask embedding e(mask) is composed of:

[0064] T={h0,...,h i ,e(mask),h i+1 ,...,h n ,e(x)} (2)

[0065] Where i refers to the index of the soft marker, h i To represent soft tokens, soft hint templates replace natural language words in handcrafted templates;

[0066] Step 2.2) In the prompt tuning method, it is crucial to map the predicted probability of label words to specific categories; in order to further improve the model performance, four different strategies are used to build a label word mapper V t , in order to fully capture the various characteristics of the expansion word; the specific details are as follows:

[0067] FastText Similarity: Compute the semantic similarity between FastText vectors and the expanded label words for each category; subsequently, select the top N words with semantic similarity exceeding 90%. a word.

[0068] Probabilistic prediction: Use a pre-trained language model such as BERT to generate a probability distribution p(·|T[MASK]) for the word "[MASK]" in the vocabulary; similar to other strategies, select the top N words with the highest semantic relevance from this probability distribution a This method can use the contextual nuances in the sentence to predict the words that best match the category relevance.

[0069] Context information: Context information is taken into account, and the expanded vocabulary is required to include the surrounding words before and after the [MASK] token; before entering BERT, [MASK] is replaced with these expanded words, and the top N words with the highest semantic relevance are selected through cross entropy loss calculation. a words;

[0070] Probability distribution optimization: The probability distribution is optimized by using the standard deviation to enhance the distinction between label words and categories. Specifically, the higher the correlation between the label word and its category, the better, and the lower the correlation with other categories, the better. This helps ensure that the label word does not make positive contributions to multiple categories, thereby reducing confusion between categories.

[0071] Step 2.3) By using the PromptForClassification function in the openprompt library, a prompt learning model PromptModel is constructed for the classification task; the PromptForClassification function accepts three parameters: the pre-trained language model plm, the template template, and the label word mapper verbalizer; the Bert model is used as the pre-trained language model, and the template prepared in step 2.1) and the label word mapper constructed in step 2.2) are assigned to these parameters accordingly to complete the construction process of the soft prompt tuning model;

[0072] Step 2.4) uses a small portion of the source domain data according to steps 1) and 2.3) to train an initial soft-hint fine-tuning model, which is then used to assign pseudo labels to instances in the target domain.

[0073] 3) Divide the target domain instances into n subsets, each of which consists of pseudo labels and instances. Select one subset for training in each iteration, and use the other subsets for testing. Obtain n labels for the target domain instances through multiple iterations, and analyze syntactic features in combination with the graph attention model.

[0074] Step 3.1) In order to fully explore the syntactic structure of Chinese text, we propose to use the graph attention network to integrate the information of adjacent nodes in the syntactic graph; combine the word dependency and part-of-speech tags to capture the syntactic pattern more comprehensively; given a Chinese sentence x = {x1,...,x i ,...,x n}, where x i represents the i-th word; using the Language Technology Platform (LTP4), each word is assigned a part-of-speech tag P = {p1,...,p i ,...,p n}, where p i represents the part-of-speech tag of the i-th word; next, a directed graph G is created to represent the syntactic structure of the Chinese text; each node in the directed graph is initialized with a randomly generated 150-dimensional feature vector to define its initial state. Syntactic structure feature f sem The calculation is as follows:

[0075] e ij =LeakyReLU(αT [Z i ‖Z j ]) (3)

[0076] l i =Elu(∑ j softmax(e ij )Z j W) (4)

[0077]

[0078] Among them, Z i represents the feature matrix of node i, α and W are learnable parameters, e ij is the attention weight between nodes i and j, l i represents the final node embedding obtained by applying the activation function to the weighted sum of adjacent node embeddings according to the attention weights, w i For each node feature l i The weight of the assignment;

[0079] Step 3.2) After the soft hint fine-tuning model initialized according to step 2.4) is trained in the source domain and tested in the target domain, each sample in the target domain is assigned a pseudo label; due to the inherent differences between the source and target domains, there are still significant challenges in accurately predicting labels, and when pseudo labels are assigned, these labels may deviate from the true values. To address this problem, an iterative soft hint fine-tuning model is introduced, which aims to improve the recognition of true labels from the predicted pseudo label pool; specifically, all samples and their corresponding pseudo labels in the target domain are initially divided into n segments; one of the segments is selected as the training dataset for the first round of hint fine-tuning model training, and the rest is used as test data for label prediction; for each instance x∈X in the target domain, where X represents the set of all instances and the template is denoted as T(·), the learning objective can be expressed as:

[0080]

[0081] After the first data segment is used to train the initial soft prompt tuning model, pseudo labels are assigned to all other instances except the instances used in training; then, based on the first model, the second model is trained using the second segment in the n partitions through an iterative process; this cycle is repeated until each segment of the n partitions is trained once; during the iterative training, the graph attention network constructed in step 3.1) is combined to perform syntactic structure feature analysis to help the model better understand Chinese text.

[0082] 4) Through the voting mechanism, the labels that are consistent in multiple iterations are selected as the final training data for model training to achieve classification prediction of Chinese adversarial text.

[0083] Step 4.1) Through this iterative process of step 3.2), the soft hint tuning model generates n pseudo labels for each instance in the target domain; these pseudo labels are then screened through a voting mechanism to identify consistent labels and determine the final label set from the intersection of all votes; the final label set is used to train the final iterative model;

[0084] Step 4.2) After training through step 4.1), the sentences in the target domain are embedded and their label categories are divided into y∈Y. In the pre-trained language model, the label word mapper V t The probability of each label word being filled in [MASK] is expressed as P([MASK] = y|x p ), so the text adversarial defense problem is regarded as a probability problem about label assignment, and its prediction probability formula is expressed as:

[0085] P(y∈Y|x)=P([MASK]=y|x p ) (7)

[0086] Step 4.3) Find V t After the predicted probability of each label word in , the predicted probability of each label word is mapped to the corresponding class label, and the average of these predicted probabilities is used for the final classification; the predicted probability of the final class label can be expressed as:

[0087] P(y|x)=f(P M |v∈V t ) (8)

[0088] Step 4.4) In the target domain sentence, given the input x t If we calculate y t = The probability of {positive} is greater than y t ={negative}, it means the sentence belongs to the positive class.

[0089] In order to test the effectiveness of the present invention, two Chinese data sets were used for testing, namely, the JD product review data set (JD) and the Ctrip hotel review data set (CT). The JD product review data set consists of comments on various products on the JD official website, and is specially constructed for the binary classification task of sentiment. About 4,000 original samples were selected as source domain data, and these samples were perturbed to form adversarial samples as target domain data. The Ctrip hotel review data set was selected from guest comments on different hotels on the Ctrip official website, and the emotions of these comments were divided into positive and negative. In the experiment, about 6,000 original comments were selected as source domain data. Similar to the JD data set, the target domain also used the same method to generate perturbation data. In order to reflect the performance of the method (DC-ISP) of the present invention, stacked denoising autoencoder (SDA), convolutional neural network (CNN), Bert, LLaMA, ChatGPT, conventional prompt tuning (Prompt), soft prompt tuning (Spot), and iterative hard prompt tuning (DC-IHP) were compared. The test results of the data set are shown in Table 1. It can be seen from Table 1 that the result accuracy index in almost all tasks is basically better than other methods.

[0090] Table 1 Accuracy of experimental results

[0091]

[0092] The present invention proposes a method for defending against adversarial attacks on Chinese text based on a soft-hint tuning model. Combining the idea of ​​domain adaptation, the original text and the perturbed text are regarded as the source domain and the target domain respectively. The initial model is trained with a small amount of source domain data to reveal the true label information of the target domain, and the soft-hint tuning model of Chinese grammatical features is enhanced through a voting mechanism combined with graph attention to achieve accurate classification; the method of the present invention effectively defends against various text attacks and improves the robustness of Chinese text classification.

[0093] The present invention is not limited to the above-mentioned embodiments. On the basis of the technical solution disclosed in the present invention, technicians in this field can make some substitutions and deformations to some technical features therein according to the disclosed technical content without creative labor, and these substitutions and deformations are all within the protection scope of the present invention.

Claims

1. A method for defending against adversarial attacks on Chinese text based on a soft prompt tuning model, characterized in that: The following steps are involved: 1) Generate target domain data using five Chinese-specific text attack strategies, and introduce source and target domains to simulate original and adversarial texts; 2) Train the initial soft hint tuning model based on a small amount of source domain data and assign pseudo labels to target domain instances; 3) Divide the target domain instances into n subsets, each of which consists of pseudo labels and instances. Select one subset for training in each iteration, and use the other subsets for testing. Obtain n labels for the target domain instances through multiple iterations, and analyze syntactic features in combination with the graph attention model. 4) Through the voting mechanism, the labels that are consistent in multiple iterations are selected as the final training data for model training to achieve classification prediction of Chinese adversarial text.

2. According to claim 1, a method for defending against adversarial attacks on Chinese text based on a soft prompt tuning model is characterized in that: The step 1) specifically includes: Step 1.1) For the original text x = {x1, x2, ..., x n }Apply the greedy search strategy to each word x i Replace with unknown token UNK, generate new text x ′ ; Then use model F to measure the decrease in confidence; The process is shown in formula (1), where y orig Represents the original label; score(x i )=1-Prob(F(x i )) yorig (1) s.t.x ′ =replace(x,x i ,UNK) Step 1.2) Sort the words in the input sentence by importance according to step 1.1); replace the words in descending order of importance until an adversarial text is generated that is sufficient to cause the model to make wrong predictions; the specific replacement strategies include the following five: Open HowNet: This strategy uses the semantic similarity provided by Open HowNet to find the top k synonyms that are semantically closest to the target word for replacement; Masked language model-based replacement: This strategy uses the pre-trained XLM-RoBERTa masked language model to predict alternative words for the target word and selects the top k words with the highest scores as candidate words; Homophone replacement: This strategy looks for characters with the same pronunciation as the target word based on the pinyin of Chinese characters, and randomly selects the first k words from the homophone candidate set for replacement; Homograph replacement: This strategy looks for Chinese characters that are visually similar to the target word based on the morphological structure of Chinese characters, and randomly selects the first k words from the lexical candidate set for replacement; Combined substitution: This strategy combines the above four strategies by sequentially substituting Open HowNet, masked language model-based substitution, homophone substitution, and homograph substitution into the target word; if the substitution successfully changes the model’s prediction, the process stops; otherwise, the strategy continues with the next most important word until the model’s prediction is successfully perturbed; After perturbation by five strategies, the original text and adversarial text are used as the source domain and target domain, respectively, thus converting the text adversarial defense task into a domain adaptation task.

3. According to claim 1, a method for defending against adversarial attacks on Chinese text based on a soft prompt tuning model is characterized in that: The step 2) specifically includes: Step 2.1) Use the automatically generated template, i.e., the soft prompt template. The labeled sentence in the source domain is a sequence of words, represented as x = {x0, ..., x i ,...,x h }; In the embedding layer of PLM, each token is mapped into a pre-trained embedding space after disambiguation, where each token corresponds to a vector of fixed size; these embedding vectors encapsulate the semantic meaning of each token and represent its contextual understanding in the entire corpus. The labeled sentence is mapped to e(x) = {e(x0),...,e(x)} via PLMs e∈M. i ),...,e(x h )}, M refers to the Chinese-RoBERTa-WM-EXT model, and the embedding e of Chinese-RoBERTa-WM-EXT is introduced as PLMs M; the soft prompt T is then composed of the embedding sequence e(x), soft label h i And the mask embedding e(mask) is composed of: T={h0,...,h i ,e(mask),h i+1 ,...,h n ,e(x)} (2) Where i refers to the index of the soft marker, h i To represent soft tokens, soft hint templates replace natural language words in handcrafted templates; Step 2.2) Four different strategies are used to build a label word mapper V t , in order to fully capture the various characteristics of the expansion words; the four different strategies include: FastText Similarity: Calculates the semantic similarity between FastText vectors and the extended label words for each category; Probabilistic prediction: Use the pre-trained language model to generate a probability distribution p(·|T[MASK]) for the word "[MASK]" in the vocabulary; select the top N words with the highest semantic relevance from this probability distribution a words; using contextual nuances in the sentence to predict the words that best fit the category; Context information: Considering context information, the expanded vocabulary is required to include the surrounding vocabulary before and after the [MASK] tag; before entering BERT, [MASK] is replaced with these expanded vocabulary, and the top N words with the highest semantic relevance are selected through cross entropy loss calculation. a words; Probability distribution optimization: optimize the probability distribution by using standard deviation to enhance the distinction between label words and categories; Step 2.3) By using the PromptForClassification function in the openprompt library, a prompt learning model PromptModel is constructed for the classification task; the PromptForClassification function accepts three parameters: the pre-trained language model plm, the template template, and the label word mapper verbalizer; the Bert model is used as the pre-trained language model, and the template prepared in step 2.1) and the label word mapper constructed in step 2.2) are assigned to the three parameters accordingly to complete the construction process of the soft prompt tuning model; Step 2.4) uses a small portion of the source domain data according to steps 1) and 2.3) to train an initial soft-hint fine-tuning model, which is then used to assign pseudo labels to instances in the target domain.

4. According to claim 1, a method for defending against adversarial attacks on Chinese text based on a soft prompt tuning model is characterized in that: The step 3) specifically includes: Step 3.1) Use the graph attention network to integrate the information of adjacent nodes in the syntactic graph and mine the syntactic structure of Chinese text. Given a Chinese sentence x = {x1,...,x i ,...,x n }, where x i Represents the i-th word; using the language technology platform, assign a part-of-speech tag P = {p1,...,p i ,...,p n }, where p i represents the part-of-speech tag of the i-th word; creates a directed graph G to represent the syntactic structure of Chinese text; each node in the directed graph is initialized with a randomly generated 150-dimensional feature vector to define its initial state; the syntactic structure feature f sem The calculation is as follows: and ij =LeakyReLU(α T [WITH i ‖WITH j ]) (3) l i =Elu(∑ j softmax(e ij )WITH j In) (4) Among them, Z i represents the feature matrix of node i, α and W are learnable parameters, e ij is the attention weight between nodes i and j, l i represents the final node embedding obtained by applying the activation function to the weighted sum of adjacent node embeddings according to the attention weights, w i For each node feature l i The weight of the assignment; Step 3.2) After the soft hint fine-tuning model initialized according to step 2.4) is trained in the source domain and tested in the target domain, each sample in the target domain is assigned a pseudo-label; an iterative soft hint fine-tuning model is introduced to improve the recognition of the true label from the predicted pseudo-label pool; all samples and their corresponding pseudo-labels in the target domain are initially divided into n segments, one of which is selected as the training dataset for the first round of hint fine-tuning model training, and the rest is used as the test data for label prediction; for each instance x∈X in the target domain, where X represents the set of all instances, the template is denoted as T(·), and the learning objective is expressed as: After the first data segment is used to train the initial soft prompt tuning model, pseudo labels are assigned to all other instances except the instances used in training; then, based on the first model, the second model is trained using the second segment in the n partitions through an iterative process; this cycle is repeated until each segment of the n partitions is trained once; during the iterative training, the syntactic structure feature analysis is performed in combination with the graph attention network constructed in step 3.1).

5. According to claim 1, a method for defending against adversarial attacks on Chinese text based on a soft prompt tuning model is characterized in that: The step 4) specifically includes: Step 4.1) Through this iterative process of step 3.2), the soft hint tuning model generates n pseudo labels for each instance in the target domain; these pseudo labels are then screened through a voting mechanism to identify consistent labels and determine the final label set from the intersection of all votes; the final label set is used to train the final iterative model; Step 4.2) After training through step 4.1), the sentences in the target domain are embedded and their label categories are divided into y∈Y. In the pre-trained language model, the label word mapper V t The probability of each label word being filled in [MASK] is expressed as P([MASK] = y|x p ), the text adversarial defense problem is regarded as a probability problem about label assignment, and the prediction probability formula is expressed as: P(y∈Y|x)=P([MASK]=y|x p ) (7); Step 4.3) Find V t After the predicted probability of each label word in , the predicted probability of each label word is mapped to the corresponding class label, and the average of these predicted probabilities is used for the final classification; the predicted probability of the final class label is expressed as: P(y | x)=f(P M | v∈V t ) (8); Step 4.4) In the target domain sentence, given the input x t If we calculate y t = The probability of {positive} is greater than y t ={negative}, it means the sentence belongs to the positive class.

Images