Robustness detection method for text classification model

Abstract

The present invention discloses a method for detecting the robustness of a text classification model, comprising: randomly replacing a target word in an original sample with a synonym to obtain a first sample; gradually optimizing the semantic similarity between the first sample and the original sample through T rounds of iteration to obtain a test sample; performing a robustness test on a target text classification model using a test sample set constructed from a number of test samples, and adjusting the training process and / or parameter configuration of the target text classification model according to the robustness test result. The present invention effectively simulates various text changes that may be encountered in the real world by generating adversarial samples that are semantically close to the original samples but slightly different in expression, and optimizing these samples through multiple rounds of iterations, thereby enabling a more comprehensive detection of the robustness of the text classification model.

Description

Technical Field

[0001] The present invention relates to the field of neural network technology, and in particular to a method for detecting the robustness of a text classification model. Background Art

[0002] With the rapid development of information technology and artificial intelligence, text classification models play an important role in many fields, such as sentiment analysis, spam detection, and automatic news classification. These models are usually based on machine learning technology, especially deep learning, to process and classify large amounts of text data.

[0003] However, although text classification models have demonstrated efficient classification capabilities in multiple application scenarios, they are often vulnerable to malicious input perturbations. This vulnerability may cause the performance of the model to drop significantly in practical applications, especially when faced with carefully designed adversarial text samples. Adversarial samples refer to data samples that have been specially processed and can mislead the model into making incorrect judgments. Therefore, improving the robustness of text classification models so that they can resist such adversarial attacks has become an important direction of current research.

[0004] Traditional model robustness enhancement methods mainly focus on the image field, while relatively few studies have been conducted on text data. In addition, existing text model robustness testing methods usually rely on white-box attacks, which means that the attacker needs to know the internal structure and parameters of the model. This requirement limits the breadth and practicality of these methods in practical applications, because in many practical situations, it is impossible for the attacker to fully understand the internal information of the target model. Summary of the invention

[0005] In order to address the deficiencies in the prior art, the present invention aims to provide a text classification model robustness detection method based on black box hard label conditions, which does not require access to the internal structure or parameters of the model and only requires the output label of the model for robustness testing.

[0006] In order to achieve the above-mentioned invention object, the technical solution provided by the present invention includes:

[0007] The robustness detection method of the text classification model includes the following steps:

[0008] S1, randomly replace the target words in the original sample with synonyms to obtain the first sample;

[0009] S2, gradually optimizing the semantic similarity between the first sample and the original sample through T rounds of iterations to obtain a test sample;

[0010] S3, using a test sample set constructed from a number of test samples to perform a robustness test on the target text classification model, and adjusting the training process and / or parameter configuration of the target text classification model according to the robustness test result;

[0011] The optimization method comprises:

[0012] S201, searching in the word embedding space for a sample that is close to the first sample as a second sample;

[0013] S202, performing disturbance optimization based on backtracking control on the second sample to obtain a third sample, and if the disturbance optimization fails, using the second sample as the third sample;

[0014] S203, performing optimal word optimization based on rule exchange on the third sample to obtain a fourth sample, and if the optimal word optimization fails, taking the sample with the highest similarity to the original sample between the first sample and the third sample as the fourth sample;

[0015] S204: Use the fourth sample as the first sample again, and repeat steps S201-S203 for T rounds to obtain a test sample.

[0016] In some preferred embodiments, the method of searching for a sample close to the first sample in the word embedding space as the second sample includes:

[0017] For each target word in the first sample, use the trained word embedding model to find at least one synonym with a close distance for the target word in the embedding space, and use the synonym to replace the corresponding target word in the first sample as the second sample.

[0018] In some preferred embodiments, the method of using the trained word embedding model to find at least one synonym with a close distance for the target word in the embedding space includes:

[0019] Using the trained word embedding model The first sample x' t Each target word w' n Convert to word vector and Where M is the dimension of word embedding;

[0020] x' t Mapping to the embedding space to get the embedding matrix E t With the perturbation matrix Add together to get the perturbed word embedding matrix Where γ is a hyperparameter that controls the intensity of the perturbation;

[0021] Will w' n Synonyms for After the synonyms in the word embedding space are mapped to the word embedding space, the L2 norm is selected. Nearest synonyms.

[0022] In some preferred embodiments, the method of performing disturbance optimization based on backtracking control on the second sample to obtain the third sample includes:

[0023] Randomly selecting a sample from the plurality of second samples as a query sample;

[0024] Determine whether the query sample is an adversarial sample of the original sample, and if not, mark the query sample as a filtered sample and exclude it from the next query;

[0025] Repeat the above steps until an adversarial sample of the original sample is found;

[0026] Perform perturbation optimization based on backtracking control on the adversarial sample to obtain a third sample.

[0027] In some preferred embodiments, the method for disturbance optimization based on backtracking control includes:

[0028] Find k words that can be restored, where Where ε is a hyperparameter that controls the perturbation optimization rate and is 0≤ε≤1, and p is the number of words currently being perturbed;

[0029] Sequentially calculating the similarity between the restored sample of each disturbed word in the third sample and the original sample;

[0030] Performing tentative restoration on the restorable words of the restored sample one by one in descending order of the similarity, if the restored sample is an adversarial sample, continuing to restore the next restorable word, if not, skipping the current word and restoring the next restorable word;

[0031] After completing the tentative restoration of all restorable words, the current sample is output as the third sample.

[0032] In some preferred embodiments, the method of performing rule-based exchange-based optimal word optimization on the third sample to obtain the fourth sample includes:

[0033] Substituting synonyms for the disturbed words in the third sample in sequence to obtain an intermediate sample;

[0034] Calculating a first similarity between the intermediate sample and the original sample, and calculating a second similarity between the third sample and the original sample;

[0035] Excluding intermediate samples whose first similarity is lower than the second similarity;

[0036] Searching for adversarial samples of the original sample in the remaining intermediate samples in descending order of the similarity;

[0037] If the search is successful, the intermediate sample is used as the fourth sample; if the search fails, the sample with the highest similarity to the original sample between the first sample and the third sample is used as the fourth sample.

[0038] Beneficial Effects

[0039] 1. Enhance the practical application robustness of the model: The present invention generates adversarial samples that are semantically close to the original samples but slightly different in expression, and optimizes these samples through multiple rounds of iterations, effectively simulating various text changes that may be encountered in the real world, thereby enabling a more comprehensive detection of the robustness of the text classification model.

[0040] 2. Wide applicability of black box testing: This method is based on the condition of black box hard label and does not need to obtain the internal structure or parameter information of the model. This testing method has high universality and practicality and is applicable to any scenario where the internal information of the model cannot be accessed externally, making this method widely applicable to a variety of commercial and security-sensitive environments.

[0041] 3. Since the present invention does not rely on an in-depth understanding of the target model, non-professionals can also use this method to evaluate and enhance the robustness of the text classification model, which lowers the technical threshold and expands its scope of application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flowchart of a method for detecting robustness of a text classification model in a preferred embodiment of the present invention;

[0043] Figure 2 It is a schematic diagram of the optimization method flow in another preferred embodiment of the present invention;

[0044] Figure 3 This is an experimental example of the present invention, in which the method proposed in the present invention is compared with the efficiency of the model trained on the BERT model for five classification tasks by other four baseline algorithms; DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention is further described below in conjunction with the accompanying drawings. In the description of the present invention, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention.

[0046] Example

[0047] like Figure 1 As shown, this embodiment provides a text classification model robustness detection method, including the steps of:

[0048] S1. Randomly replace the target words in the original sample with synonyms to obtain the first sample.

[0049] Under the black box hard label test condition, it is impossible to obtain the decision score of the tested text classification model, so it is impossible to calculate the degree of influence of each word in the test sample on the tested text classification model. Therefore, the tester can only generate the initial test sample by randomly replacing some words. At this time, the purpose of initializing the first sample (initial test sample) is to convert the original sample into the first sample by adding random noise. It should be understood that the content words of the input text during text classification have a greater impact on the semantics of the text, and the function words have a smaller impact on the semantics. The content words refer to words with clear semantic content, such as adjectives, adverbs, nouns and verbs, etc. The corresponding concept is a function word, which refers to a word that has no specific meaning and is mainly used to express a grammatical relationship, such as a preposition, conjunction, auxiliary word, etc. Therefore, this embodiment needs to perform a part of speech check on the words of the original sample and screen the content words as the target words. It should be noted that the words operated in all subsequent steps are all content words screened out from the sample.

[0050] S2. The semantic similarity between the first sample and the original sample is gradually optimized through T rounds of iteration to obtain a test sample. Considering that the semantic similarity with the original sample is reduced due to a large number of words being changed when the first sample is generated using only the random initialization method, the present invention limits the number of iterations to T rounds.

[0051] Specifically, Figure 2 As shown, the optimization method includes:

[0052] S201, searching for a sample with a close distance to the first sample as a second sample in the word embedding space. The word embedding space refers to a high-dimensional space that converts words into numerical vectors. In this space, each word is represented as a mathematical vector that captures the semantic and grammatical properties of the word. With this representation, the computer can understand and process natural language data. Such conversion is usually completed by a word embedding model, the core idea of ​​which is to convert words into vectors, the positions and distances of which in the vector space can reflect the semantic and grammatical relationships between words. For example, semantically similar words, such as "university" and "college", will also have similar vectors in the word embedding space. Since this part of the content is not the focus of protection of the present invention, those skilled in the art can select from the commonly used word embedding models in the prior art, such as Word2Vec, GloVe and FastText, as needed, and the present invention does not make further limitations.

[0053] In some preferred embodiments, a method for searching for a sample close to the first sample in the word embedding space as the second sample is provided, which specifically includes:

[0054] For each target word in the first sample, use the trained word embedding model to find at least one synonym with a close distance for the target word in the embedding space, and use the synonym to replace the corresponding target word in the first sample as the second sample. Although the size of the search space by searching for neighboring samples through word embedding is much smaller than the size of all possibilities, it is still difficult to traverse all possible samples. In other preferred embodiments, the entire search space can be represented by limiting the number of sampling times to δ times, which will generate a smaller second sample set.

[0055] Furthermore, in some preferred embodiments, a preferred method for using a trained word embedding model to find at least one synonym with a similar distance for a target word in the embedding space is provided, specifically including:

[0056] S101. Using the trained word embedding model The first sample x' t Each target word w' n Convert to word vector and Where M is the dimension of word embedding;

[0057] S102, x' t Mapping to the embedding space to get the embedding matrix E t With the perturbation matrix Add together to get the perturbed word embedding matrix Where γ is a hyperparameter that controls the perturbation intensity. It should be understood that U t Each element in is sampled from a standard Gaussian distribution.

[0058] S103, w' n Synonyms for After the synonyms in the word embedding space are mapped to the word embedding space, the L2 norm is selected. The nearest synonym. The L2 norm, also known as the Euclidean norm or the Euclidean norm, is one of the most common vector norms and is used to measure the size of vector elements. Specifically, the L2 norm refers to the distance from the origin to a point in the n-dimensional Euclidean space. In this embodiment, the L2 norm refers to the distance between the synonym and the perturbed word in the word embedding space.

[0059] S202. Perform perturbation optimization based on backtracking control on the second sample to obtain a third sample. If the perturbation optimization fails, the second sample is used as the third sample. The backtracking is to "retrace" or "withdraw" during the perturbation process, that is, when the latest changes fail to achieve the expected effect or cause quality degradation, these changes can be undone. If the optimization of this step fails, skip the optimization step and use the input of this step directly as the input of the next step. In the present invention, this means that the perturbation in the sample can be gradually adjusted to control the quality and adversarial nature of the generated sample. The specific operation steps are as follows:

[0060] Step 1: Evaluate the effect of the perturbation

[0061] Apply natural language processing tools and models to evaluate the effect of the perturbed samples. Mainly check whether the samples retain the original semantics and bring the expected challenges to the target classification model.

[0062] Step 2: Retrospective Adjustment

[0063] If the perturbed sample fails to achieve the expected adversarial effect or affects the naturalness and semantic coherence of the sample, backtracking is used to undo some or all of the perturbations. This step requires multiple iterations to find the optimal perturbation balance by fine-tuning the degree and type of perturbations.

[0064] Step 3: Optimize the perturbation

[0065] After multiple rounds of backtracking and adjustment, the final perturbation strategies are determined, which can maximize the adversarial nature of the samples without destroying the naturalness of the text.

[0066] For this embodiment, the method of performing disturbance optimization based on backtracking control on the second sample to obtain the third sample specifically includes:

[0067] S2021. Randomly select a sample from a plurality of the second samples as a query sample;

[0068] S2022, determining whether the query sample is an adversarial sample of the original sample, and if not, marking the query sample as a filtered sample and excluding it from the next query;

[0069] S2023. Repeat the above steps S2021-S2022 until an adversarial sample of the original sample is found;

[0070] S2024. Perform perturbation optimization based on backtracking control on the adversarial sample to obtain a third sample.

[0071] It should be understood that usually not all disturbances in the sample are meaningful, and there are meaningless disturbances. Moreover, since the disturbed words will not participate in the subsequent optimization after being replaced with the original words, too much word disturbance optimization will fall into the local optimum. In order to solve this technical problem, in some preferred embodiments, a method of disturbance optimization based on backtracking control is provided, which specifically includes:

[0072] S20241. Find k words that can be restored, where Where ε is a hyperparameter that controls the perturbation optimization rate and is 0≤ε≤1, and p is the number of words currently being perturbed; It is a rounding function, which means that the value inside it is rounded up to the nearest integer.

[0073] S20242, sequentially calculating the similarity between the restored sample of each disturbed word in the third sample and the original sample;

[0074] S20243, tentatively restoring the restorable words of the restored sample one by one in descending order of the similarity, if the restored sample is an adversarial sample, continuing to restore the next restorable word, if not, skipping the current word and restoring the next restorable word;

[0075] S20244. After completing the tentative restoration of all restorable words, output the current sample as the third sample.

[0076] S203, optimizing the third sample by the best word based on rule exchange to obtain a fourth sample. If the optimization of the best word fails, the sample with the highest similarity to the original sample among the first sample and the third sample is used as the fourth sample. Those skilled in the art should know that restoring words can significantly improve the similarity of adversarial samples, but the combination of different words still has a certain impact on their semantic similarity. In some preferred embodiments, a greedy search method is provided to find the best word and replace it in the third sample. The specific method is as follows:

[0077] S2031, performing synonym replacement on the disturbed words in the third sample in sequence to obtain an intermediate sample;

[0078] S2032, calculating a first similarity between the intermediate sample and the original sample, and calculating a second similarity between the third sample and the original sample;

[0079] S2033, excluding intermediate samples whose first similarity is lower than the second similarity;

[0080] S2034, searching for adversarial samples of the original sample in the remaining intermediate samples in descending order of the similarity;

[0081] S2035. If the search is successful, use the intermediate sample as the fourth sample; if the search fails, use the sample with the highest similarity to the original sample between the first sample and the third sample as the fourth sample.

[0082] S204: Use the fourth sample as the first sample again, repeat the above optimization steps for T rounds to obtain a test sample.

[0083] S3. Use a test sample set constructed from several test samples to perform a robustness test on the target text classification model, and adjust the training process and / or parameter configuration of the target text classification model according to the robustness test results. Specifically, the test sample set is input into the target text classification model, and the classification results of the model on these samples are observed. The key is to compare the response differences of the model to the test sample set and to the original data set. Through this comparison, the performance stability and accuracy of the model when encountering unseen or intentionally disturbed inputs can be evaluated.

[0084] Specifically, the evaluation and analysis can be divided into two aspects:

[0085] 1. Performance evaluation: Analyze the model’s key performance indicators such as classification accuracy and misclassification rate for test samples. Check whether the model’s performance degrades significantly due to slight changes in the input.

[0086] 2. Error analysis: Analyze in detail the test samples that the model misclassified to identify the model’s weaknesses, such as sensitivity to specific types of perturbations or certain specific semantics.

[0087] According to the robustness test results, adjust the model's training process and parameter configuration. Possible adjustments include but are not limited to:

[0088] 1. Retraining: Retrain the model using an expanded training set containing test samples to enhance its ability to handle similar perturbations.

[0089] 2. Parameter optimization: adjust the learning rate, regularization parameters, etc. to optimize the model's tolerance to disturbances.

[0090] 3. Algorithm adjustment: adopt a more robust algorithm or increase the complexity of the model, such as introducing adversarial training technology.

[0091] Through this series of steps, the present invention can not only effectively evaluate the robustness of existing text classification models, but also provide specific improvement strategies to improve the reliability and security of the models in practical applications. This is of great significance for dealing with potential malicious attacks or natural fluctuations in input data.

[0092] Experimental example

[0093] 1. Dataset

[0094] This experiment includes the following five classification datasets and three NLI datasets: (1) MR: Movie review sentiment binary classification dataset. (2) AG's News: News type classification dataset. Contains four categories, namely world, sports, business and science. (3) Yahoo: Question and answer type classification dataset. Contains ten categories. (4) Yelp: Sentiment binary classification dataset. (5) IMDB: Movie review sentiment binary classification dataset. (6) SNLI: NLI dataset from Amazon Mechanical Turk1. (7) MNLI: NLI dataset from multiple sources. (8) mMNLI: NLI dataset with inconsistent sources from MNLI.

[0095] 2. Target text classification model

[0096] The commonly used models for NLP classification tasks are: 1. WordCNN; 2. WordLSTM; 3. BERT. For the fairness of the experiment, the experiment uses the model data disclosed in the HLGA paper (https: / / github.com / RishabhMaheshwary / hard-label-attack).

[0097] 3. Experimental environment

[0098] To ensure the fairness of the experiment, the experiment is repeated on the same dataset. All experiments are conducted on NVIDIA Tesla A10 GPU using PyTorch framework. The word embedding space is represented as The widely used CounterFitting Word Vectors model is used. Each word is associated with a set of 50 synonyms, which is consistent with the baseline method. The hyperparameter γ is set to 0.5 to adjust the strength of the perturbation matrix. The number of word embedding space searches is set to δ=300. In the controlled backtracking perturbation, the hyperparameter ε=0.3 controls the perturbation optimization rate. The baseline is reproduced using publicly available code.

[0099] 4. Baseline Method

[0100] Three currently optimal baseline methods were selected for experimental comparison, all of which were proposed in the past three years. The results of the comparison with these advanced methods prove that the method proposed in this invention (hereinafter referred to as TextSwindler) is superior to the existing technology. The three methods compared are:

[0101] (1) HLGA: This method uses a genetic algorithm to optimize the semantic similarity of adversarial samples. First, a large number of adversarial sample candidate sets are generated, and then crossover, mutation, and selection operations are applied to identify the optimal candidate samples to achieve adversarial sample generation.

[0102] (2) LeapAttack, after random initialization, optimizes adversarial samples by continuously moving samples closer to the decision boundary, estimating gradients, and searching for suitable words.

[0103] (3) SSPAttack: This method follows a simple rule-based exchange paradigm during the optimization process, breaking the optimization process into two steps: first reducing the perturbation rate, and then replacing the replaced synonyms in the direction of the original word.

[0104] (4) HQA-Attack, this method simplifies the generation of black-box hard-label adversarial text into two steps: 1. Restoring the changed words to the original words to minimize the perturbation rate, and 2. Refining the samples using synonym sets to enhance semantic similarity.

[0105] 5. Evaluation Metrics

[0106] The evaluation indicators mainly refer to four aspects: attack success rate, perturbation rate, semantic similarity and number of queries.

[0107] 1) Post-attack accuracy (Acc): The classification accuracy of the model after the attack algorithm is used to attack. It represents the effectiveness of the attack algorithm in generating adversarial samples under the experimental setting. The lower the value, the stronger the attack capability. In the detection of model robustness, this indicator reflects the classification accuracy of the model after the sample generated by the test algorithm is tested. It is used to measure the robustness of the model when facing the test sample. The higher the value, the stronger the robustness of the model.

[0108] 2) Semantic similarity (Sim): The quality evaluation of adversarial examples depends largely on semantic similarity. To calculate semantic similarity, this experimental example uses the USE model to calculate semantic similarity as a measurement indicator to measure the semantic difference between the generated adversarial samples and the original samples. In the detection of model robustness, this indicator is used to evaluate the degree of semantic preservation of the sample after being processed by the test algorithm. The higher the value, the closer the semantics are to the original sample, thus reflecting the performance of the test algorithm.

[0109] 3) Perturbation rate (Pert): The key to adversarial attacks is to induce the model to make wrong judgments with smaller perturbations. The perturbation rate is calculated by dividing the number of words changed in the adversarial example by the total number of words. The lower the value, the better the performance of the algorithm. In the detection of model robustness, this indicator measures the degree of variation of the test sample. The lower the value, the more subtle the change of the sample, which can better test the sensitivity of the model to small variations.

[0110] 4) Query number (NQ): Under the hard label black box condition, in order to prevent the service provider from limiting the number of API accesses, the query number of the model is used as an evaluation indicator to reflect the attack efficiency of the algorithm. The lower the value, the higher the efficiency of the algorithm. In the detection of model robustness, the number of sample queries required to evaluate the model is recorded. This indicator reflects the efficiency of the test algorithm. The lower the value, the less resources and time the algorithm needs to evaluate the robustness of the model.

[0111] 6. Experimental results

[0112] 6.1 Robust Performance Analysis

[0113] The goal of this project is to find a method that can effectively evaluate the robustness of text classification models by generating test samples with high semantic similarity and low perturbation rate while ensuring that the test sample set covers a wide range. Table 1 shows the results of TextSwindler and baseline methods for text classification tasks. Due to the different initialization methods, TextSwindler obtains higher post-attack accuracy (the values ​​in brackets are the original accuracy). Table 2 shows the semantic similarity and perturbation rate when evaluating the natural language inference model (BERT), with the best results shown in bold.

[0114] Table 1 Comparative evaluation results of TextSwindler, HLGA, LeapAttack and SSPAttack in text classification tasks

[0115]

[0116]

[0117] Table 2 Algorithms attacking NLI tasks

[0118]

[0119] TextSwindler performs well in model robustness testing by generating test samples with high semantic similarity and low perturbation rate. As shown in Table 1, when evaluating text classification models for BERT, WordCNN, and WordLSTM, TextSwindler improves semantic similarity by 2.83%, 2.16%, and 4.33% on the short text datasets MR, AG, and Yahoo, respectively, compared with the optimal baseline SSPAttack, while reducing the perturbation rate by 0.840%, 0.510%, and 0.613%, respectively. On the long text datasets Yelp and IMDB, TextSwindler achieves a 3.1% and 2.37% semantic similarity improvement compared to SSPAttack, and the perturbation rate is reduced by 1.020% and 0.873%, respectively.

[0120] In addition, for the natural language inference models of the NLI datasets SNLI, MNLI, and mMNLI, TextSwindler achieved an improvement of 4.3%, 4.9%, and 3% in semantic similarity compared to SSPAttack, and a reduction of 1.79%, 1.83%, and 2.45% in perturbation rate, respectively, as shown in Table 2. These improvements are attributed to the global replacement initialization method adopted by TextSwindler, which considers all replaceable words in the subsequent optimization process, effectively reducing the variability of the samples while maintaining highly consistent semantics with the original text. In addition, the use of a perturbation optimization strategy based on backtracking control is similar to adjusting the learning rate in deep learning training, which effectively controls the appropriate decline in the test perturbation rate and improves the convergence efficiency and quality of the algorithm.

[0121] 6.2 Test efficiency analysis

[0122] In addition to evaluating the quality of generated test samples, the robustness test of text classification models also needs to focus on test efficiency, that is, the number of times the target model is queried. Under actual conditions, for example, NLP platforms based on deep learning (such as Alibaba's NLP platform) usually limit the number of times users visit different interfaces per day. Therefore, using a more efficient test sample generation algorithm can evaluate the model's ability to handle high-quality test samples under the limited number of queries. Figure 3 As shown in the figure, the experiment was tested on five classification datasets: MR, AG, Yahoo, Yelp, and IMDB, and the target model was BERT. Since the text length distribution of the NLI dataset is similar to that of MR, MR is used as the NLI short text dataset as the experimental object. Among them, TextSwindler generated the highest quality test samples with fewer queries.

[0123] In addition, as shown in Table 3, for evaluating the BERT model trained on the five datasets of MR, AG, Yahoo, Yelp, and IMDB, TextSwindler reduces the number of queries required for algorithm convergence by 32.8%, 43.9%, 36.4%, 45.5%, and 59.7% compared to the baseline method SSPAttack with the best quality of generating test samples. As shown in Table 3, compared with the fastest convergence baseline method LeapAttack, although LeapAttack converges faster than TextSwindler when evaluating the Yahoo dataset, Figure 3 It can be seen that TextSwindler can generate test samples with higher semantic similarity and lower perturbation rate under the same conditions.

[0124] Table 3 Number of queries required for algorithm convergence

[0125]

[0126] In summary, under the same number of queries, TextSwindler shows more efficient and high-quality test performance compared with the baseline method. This superior performance is attributed to the combination of the method of finding neighboring samples based on the word embedding space and the method based on rule replacement. By generating diverse neighboring test samples in the word embedding space, the problem of lack of diversity in test samples due to lack of randomness caused by methods that rely solely on rule-based replacement is alleviated. Since the generated test samples are more diverse, the subsequent backtracking perturbation is more likely to be combined with the rule-based replacement method, which will reduce the perturbation rate and increase its semantic similarity, thereby improving the efficiency of the algorithm in model robustness testing.

[0127] 6.3 Robustness Testing Based on Large Language Model

[0128] In many natural language processing tasks, large language models (LLMs) such as OpenAI's ChatGPT and Alibaba's QWen model have shown excellent performance. In order to evaluate the robustness of these models in small sample classification tasks, these models were tested using the TextSwindler method. Given the limit on the number of API calls, 100 samples from the MR dataset were randomly selected and tested under the limit of a maximum of 1,000 queries.

[0129] As shown in Table 6, after robustness testing using TextSwindler, although these large language models originally showed high accuracy on the few-shot classification task, the test results show that their classification accuracy dropped by 67% and 72%, respectively. This result shows that although large models perform well on standard tasks, their robustness still needs to be improved when facing test samples that are carefully designed to simulate the challenges in potential real-world applications.

[0130] This robustness test based on TextSwindler can not only evaluate the stability of large language models when dealing with adversarial samples, but also reveal aspects of the model that need to be further enhanced before actual deployment. This has important guiding significance for the development of more robust NLP systems, helping model developers and researchers understand and improve the model's response strategies when facing complex inputs.

[0131] Table 4 Comparison of attack performance on large language models

[0132]

[0133]

[0134] 6.4 Parameter Analysis

[0135] This experimental example statistically analyzes the impact of controlling the backtracking optimization perturbation rate on the model robustness test performance. To ensure generalization, the experiment evaluates the robustness of the BERT model after training on five text classification datasets, with the perturbation rate set ranging from 0.1 to 0.5. As shown in Table 5, the performance of the model robustness test (semantic similarity and perturbation rate) is not significantly affected by the perturbation rate. This means that TextSwindler can always produce stable test results under different settings when controlling the backtracking optimization perturbation rate. In addition, Table 5 shows that setting the perturbation rate to 0.3 helps generate higher quality test samples. Therefore, this experimental example sets the perturbation rate to 0.3 in other experiments.

[0136] Table 5 Parameter analysis

[0137]

[0138] Through this parameter analysis, we can not only understand the impact of the perturbation rate setting on the model test performance, but also optimize the test method to ensure that stable and high-quality test results can be obtained when evaluating the robustness of the model. The optimization of this method helps to more accurately evaluate and improve the robustness of the text classification model in practical applications, ensuring that the model maintains efficient and accurate performance when facing different degrees of input perturbations.

[0139] 6.5 Ablation Experiment

[0140] This experiment example conducts ablation experiments on the proposed global replacement, neighboring sample search based on word embedding space, and perturbation optimization based on backtracking control to evaluate the role of these methods in improving the robustness test of the BERT model. The experiment tests the BERT model trained with the AG and IMDB datasets. The results are shown in Table 6, which include three parts:

[0141] 1. Local replacement: This experimental example evaluates the impact of traditional initialization methods and the proposed global initialization method on test performance. In the experiment, local replacement initialization is evaluated instead of global replacement initialization to observe the impact of the initialization range on the test effect.

[0142] 2. No search for neighboring samples: This experimental example evaluates the change in model robustness without using the search for neighboring samples based on the word embedding space. The step of searching for neighboring samples based on the word embedding space is removed in the experiment to determine the contribution of this step to improving the model's ability to resist different types of inputs.

[0143] 3. Uncontrolled perturbation backtracking: This experimental example evaluates the robustness of the model without controlled perturbation backtracking. The experiment achieves uncontrolled backtracking perturbation by setting the hyperparameter to 1 to test the importance of controlled perturbation backtracking for optimizing the quality of test samples and model robustness.

[0144] Table 6 Ablation experiment

[0145]

[0146] Through these ablation experiments, we can clarify the role and importance of each component in the entire model robustness testing framework. This helps to further understand and optimize the robustness testing method, ensuring that the text classification model's ability to handle complex inputs can be effectively evaluated and improved in practical applications. These experimental results show the relative importance and role of each technical component when preparing the model to face real-world data.

[0147] As shown in Table 7, it can be seen from the ablation experiment that:

[0148] 1. Impact of local replacement initialization:

[0149] Traditional local replacement initialization may generate test samples with fewer perturbed words, but at the same time reduce their semantic similarity. This indicates that the traditional local replacement method may ignore more important words, resulting in the failure to fully challenge the model's processing capabilities when testing the model's robustness, and it is easy to fall into a local optimal solution.

[0150] 2. The impact of deleting the search neighboring samples:

[0151] When the step of searching for neighboring samples is removed from the test algorithm, the number of model queries required increases significantly (to 1.67-3.34 times the original), the semantic similarity of the generated test samples decreases by 9.9%-2.2%, and the number of perturbed words increases by 1.29%-3.58%. This shows that using a method based on word embedding space to generate diverse test samples helps optimize the efficiency and quality of model robustness testing, reduce unnecessary perturbations, and maintain higher semantic similarity.

[0152] 3. Experiments with uncontrolled backtracking disturbances:

[0153] Without the implementation of controlled backtracking perturbation optimization, the semantic similarity of the generated test samples decreased by 1.3%-2.2%. This shows that the traditional method may ignore the impact of word importance on the quality of test samples, while the perturbation optimization based on backtracking control can effectively increase the possibility of retaining important words, thereby improving the quality of test samples.

[0154] The results of these ablation experiments not only reveal the role and importance of each component in improving the robustness test of text classification models, but also help us understand how to select and configure these technical components to optimize the test results when designing robustness tests. Through these insights, we can further optimize the robustness test methods to ensure that the text classification model can effectively evaluate and improve its processing capabilities for various inputs in practical applications.

[0155] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A text classification model robustness detection method, characterized in that: Includes steps: S1, randomly replace the target words in the original sample with synonyms to obtain the first sample; S2, through Iteratively optimizing the semantic similarity between the first sample and the original sample step by step to obtain a test sample; S3, using a test sample set constructed from a number of test samples to perform a robustness test on the target text classification model, and adjusting the training process and / or parameter configuration of the target text classification model according to the robustness test result; The optimization method comprises: S201, searching in the word embedding space for a sample that is close to the first sample as a second sample; S202, performing disturbance optimization based on backtracking control on the second sample to obtain a third sample, and if the disturbance optimization fails, using the second sample as the third sample; S203, performing optimal word optimization based on rule exchange on the third sample to obtain a fourth sample, and if the optimal word optimization fails, taking the sample with the highest similarity to the original sample between the first sample and the third sample as the fourth sample; S204: Use the fourth sample as the first sample again, and repeat steps S201-S203 for T rounds to obtain a test sample.

2. The text classification model robustness detection method according to claim 1, characterized in that: The method of searching for a sample close to the first sample in the word embedding space as the second sample comprises: For each target word in the first sample, use the trained word embedding model to find at least one synonym with a close distance for the target word in the embedding space, and use the synonym to replace the corresponding target word in the first sample as the second sample.

3. The text classification model robustness detection method as claimed in claim 2, characterized in that: The method of using the trained word embedding model to find at least one synonym with a close distance for the target word in the embedding space includes: Using the trained word embedding model The first sample Each target word in Convert to word vector and ,in is the dimension of word embedding; Will Mapping to the embedding space to get the embedding matrix ;Will With the perturbation matrix Add together to get the perturbed word embedding matrix , ,in is a hyperparameter that controls the intensity of the perturbation; Will Synonyms for After the synonyms in are mapped to the word embedding space, Norm selection and Nearest synonyms.

4. The text classification model robustness detection method according to claim 1, characterized in that: The method for performing disturbance optimization based on backtracking control on the second sample to obtain the third sample includes: Randomly selecting a sample from the plurality of second samples as a query sample; Determine whether the query sample is an adversarial sample of the original sample, and if not, mark the query sample as a filtered sample and exclude it from the next query; Repeat the above steps until an adversarial sample of the original sample is found; Perform perturbation optimization based on backtracking control on the adversarial sample to obtain a third sample.

5. The text classification model robustness detection method as claimed in claim 4, characterized in that: The method for disturbance optimization based on backtracking control includes: Find k words that can be restored, where ;in, To control the perturbation optimization rate hyperparameters and , is the number of words currently disturbed; Sequentially calculating the similarity between the restored sample of each disturbed word in the third sample and the original sample; Performing tentative restoration on the restorable words of the restored sample one by one in descending order of the similarity, if the restored sample is an adversarial sample, continuing to restore the next restorable word, if not, skipping the current word and restoring the next restorable word; After completing the tentative restoration of all restorable words, the current sample is output as the third sample.

6. The text classification model robustness detection method according to claim 1, characterized in that: The method of performing rule-based exchange-based optimal word optimization on the third sample to obtain the fourth sample comprises: Substituting synonyms for the disturbed words in the third sample in sequence to obtain an intermediate sample; Calculating a first similarity between the intermediate sample and the original sample, and calculating a second similarity between the third sample and the original sample; Excluding intermediate samples whose first similarity is lower than the second similarity; Searching for adversarial samples of the original sample in the remaining intermediate samples in descending order of the similarity; If the search is successful, the intermediate sample is used as the fourth sample; if the search fails, the sample with the highest similarity to the original sample between the first sample and the third sample is used as the fourth sample.

Images