Perturbation mode incremental toxicity data continuous detection method and system

Abstract

The invention discloses a perturbation mode incremental toxicity data continuous detection method and system, and the method comprises the steps: obtaining to-be-detected texts, carrying out the perturbation of different modes on all to-be-detected texts through employing a toxicity data jailbreak perturbation frame, obtaining jailbreak to-be-detected difficult texts in each perturbation mode, constructing a data set in each perturbation mode, and carrying out the detection of the to-be-detected difficult texts in each perturbation mode; the data set is divided into a training set, a verification set and a test set; constructing a toxicity continuous detection model based on feature playback, training the toxicity continuous detection model by adopting the training set under each disturbance mode, and verifying by adopting the verification set to obtain an optimal toxicity continuous detection model; and sending the test set in each disturbance mode into the optimal toxicity continuous detection model for detection to obtain a corresponding toxicity detection result. According to the method, a jailbreak disturbance framework is provided to generate data sets in different disturbance modes, and a continuous learning method based on feature playback is provided, so that the continuous detection capability of toxic texts in different disturbance modes is effectively promoted.

Description

Technical Field

[0001] The present invention belongs to the technical field of text toxicity auditing, and in particular relates to a method and system for continuous detection of toxicity data with increasing disturbance patterns. Background Art

[0002] With the rapid development of the Internet, the rapid spread of social media, frequent online communication, and the increasing pressure on online content review, it is becoming increasingly important to automatically detect whether text is toxic. Text toxicity refers to the presence of rude and impolite remarks, discrimination against individuals or groups, and other negative views, which will make the online communication environment unstable and unsafe, and have serious negative effects on society and individuals. Therefore, the task of text toxicity detection aims to automatically determine whether the text is toxic, promptly curb the release and further spread of toxic texts, and maintain a safe and stable online social environment.

[0003] The current mainstream toxicity detection methods mainly focus on improving the ability to detect toxic texts in normal organizational forms. However, since most toxic texts are formed for malicious purposes by users, users will use text perturbation methods such as homograph replacement, character duplication, order disturbance, and replacement according to the performance of the toxicity detector to perturb the toxic text, thereby avoiding detection and jailbreaking the toxicity detector. Although some methods add single or partial perturbed data to the detector training to improve the robustness of the detector, they still cannot handle the increasingly changing and increasing jailbreak perturbed toxic texts. Considering that the continuous learning paradigm aims to enable the model to have incremental learning capabilities for continuously added categories, fields, and tasks, continuous learning is applied to toxicity detection under dynamic jailbreak perturbation mode to form a continuous toxicity detection model for different jailbreak perturbations. Improving the continuous robustness of toxicity detection has become the research focus of this method. Summary of the invention

[0004] One purpose of the present invention is to address the shortcomings of the prior art and provide a method for continuous detection of toxic data with increasing disturbance patterns. The method proposes a jailbreak perturbation framework to generate data sets under different disturbance patterns, and proposes a continuous learning method based on feature playback, which effectively promotes the continuous detection capability of toxic text under different disturbance patterns.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A method for continuous detection of toxicity data with increasing disturbance mode comprises the following steps:

[0007] Step 1: Get the text to be detected, use the toxic data jailbreak perturbation framework to perturb all the texts to be detected in different modes, obtain the jailbreak difficult text to be detected under each perturbation mode, and use this to construct the data set under each perturbation mode, and divide the data set into training set, verification set and test set;

[0008] Step 2: Construct a toxicity continuous detection model based on feature playback, use the training sets under each disturbance mode to train the toxicity continuous detection model and use the validation set to verify it, and obtain the optimal toxicity continuous detection model;

[0009] Step 3: Send the test sets under each perturbation mode into the optimal toxicity continuous detection model for detection to obtain the corresponding toxicity detection results.

[0010] Furthermore, in step 1, the toxic data jailbreak perturbation framework includes a selector, a perturbator and a verifier; wherein the selector selects toxic words in the text to be tested by adopting multiple strategies, the perturbator perturbs the words selected by the selector in different modes, and the verifier filters out non-toxic text from the perturbed text.

[0011] Furthermore, the method of selecting toxic words by the selector includes:

[0012] Select the toxic text;

[0013] For the selected toxic text, select toxic words based on the PerspectiveAPI;

[0014] Obtain the toxicity score of the toxic text based on the PerspectiveAPI, and select toxic words that increase the toxicity score of the toxic text;

[0015] The mutual information MI-based measurement method measures the false correlation between toxic words and toxic tags, and sets the false correlation threshold to the sum of the mean and standard deviation of MI. When MI is greater than the threshold, the word is considered to have a false correlation and is selected. Otherwise, it is not selected.

[0016] Furthermore, the perturbation mode of the perturbator on the word includes:

[0017] Insert perturbation, randomly select special characters from the word selected by the selector and insert them into the word;

[0018] Deletion perturbation, randomly selecting some characters in the word selected by the selector for deletion;

[0019] Repeat perturbation, randomly select some characters in the word selected by the selector and repeat them;

[0020] Swap perturbation, randomly select two characters in the word selected by the selector and swap their order;

[0021] Homograph perturbation, randomly selects some characters in the word selected by the selector and replaces them with other characters that are visually similar;

[0022] Masked word perturbation randomly selects some characters in the words selected by the selector and replaces them with special characters;

[0023] Abbreviation perturbation, replacing relevant words or phrases in the text with abbreviations or slang;

[0024] Irrelevant disturbance: randomly select non-toxic words to form prefixes and add them before the text to be tested;

[0025] Authoritative perturbations, creating prompts that require the large model to play the role of different authoritative experts and generate first-person self-introductions.

[0026] Furthermore, the toxicity continuous detection model includes a toxicity detection module, a playback sample selector module, a feature playback loss function module, and a toxicity detection classification loss function module.

[0027] Furthermore, the toxicity detection module includes a pre-trained language model BERT based on the Transformer structure and a classifier for detecting sample toxicity. The pre-trained language model is used to convert text characters into feature vectors that can be understood and subsequently processed by the model. The classifier for detecting sample toxicity is used to predict the toxicity classification of the sample, namely:

[0028] y=Classifier(E)

[0029] Among them, E is the feature vector extracted by the pre-trained language model BERT, and Classifier is responsible for assigning label y to the text to be tested.

[0030] Furthermore, the playback sample selector module is used to calculate the Euclidean distance between the corresponding feature vectors of different texts based on the feature vector set E of each text extracted by the pre-trained language model BERT in the toxicity detection module, so as to select the typical playback sample with the shortest distance to all samples:

[0031] E=BERT(D)

[0032] distance=L2(E i ,E j )

[0033] Among them, E i They represent the feature vector of the i-th sample to be tested in the test text set D, E j They respectively represent the feature vector of the jth test sample in the test text set D, and L2 represents the Euclidean distance between the two vectors.

[0034] Furthermore, after the playback sample is selected by the playback sample selector, the feature playback loss loss function module uses a negative logarithmic function to calculate the feature similarity between the features extracted by the detection model at the current moment and the previous moment of the playback sample as the feature playback loss:

[0035]

[0036] in, They represent the features extracted by the detector of the i-th playback sample at time j-1, × represents the matrix product, They represent the features extracted by the detector of the i-th playback sample at time j, and diag represents the diagonal elements of the product, which represents the feature similarity of the playback samples.

[0037] Furthermore, the toxicity detection classification loss module is used to calculate the sum of the toxicity detection task loss and the continuous learning task loss, wherein the cross entropy loss function is used to calculate the loss between the true 0-1 label and the predicted distribution of the toxicity continuous detection model as the loss of the toxicity detection task:

[0038]

[0039] Where, L cls is the loss y of the toxicity detection task i represents the true toxicity label of the sample, represents the toxicity label predicted by the toxicity continuous detection model;

[0040] The feature replay loss is used as the loss of the continuous learning task, and the sum of the toxicity detection task loss and the continuous learning task loss is used as the objective function during training:

[0041] L=L cls +L kl,f .

[0042] Another object of the present invention is to provide a system for implementing the above-mentioned method for continuously detecting toxicity data with increasing disturbance mode, comprising:

[0043] The data set preparation module is used to obtain the text to be detected, and then use the toxic data jailbreak perturbation framework to perturb all the texts to be detected in different modes, so as to obtain the jailbreak difficult text to be detected under each perturbation mode, and thereby construct the data set under each perturbation mode, and divide the data set into training set, validation set and test set;

[0044] The model building training module is used to build a toxicity continuous detection model based on feature playback, and use the training set under each disturbance mode to train the toxicity continuous detection model and use the validation set to verify it, so as to obtain the optimal toxicity continuous detection model;

[0045] The toxicity detection module is used to send the test sets under each disturbance mode into the optimal toxicity continuous detection model for detection to obtain the corresponding toxicity detection results.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] 1) This paper proposes a toxicity data jailbreak perturbation framework, in which three core modules, namely, selector, perturbator and verifier, are designed. Based on this framework, toxic texts with existing normal organizational forms are perturbed. Four toxicity sample selection strategies and nine jailbreak perturbation strategies are used during perturbation to generate a dataset with high toxicity diversity and high perturbation diversity, which is then applied to toxicity detection to fill the dataset missing problem under this research problem.

[0048] 2) The present invention proposes a continuous toxicity detection model based on feature playback, so that the toxicity detection model can retain the knowledge of existing disturbance patterns, while expanding the features of newly added disturbance pattern toxicity data, improving the detection ability of toxic texts of different new and old disturbance patterns, and effectively improving the robustness of the toxicity detector in continuous toxicity detection under different jailbreak disturbances. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flow chart of a method for continuous detection of toxicity data with increasing disturbance mode according to an embodiment of the present invention;

[0050] Figure 2 A data generation framework diagram for an embodiment of the present invention;

[0051] Figure 3 It is a schematic diagram of the continuous toxicity detection model detection according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present invention will be described clearly and completely below in combination with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0054] The present invention will be further described below in conjunction with specific embodiments, but the present invention is not limited thereto.

[0055] like Figure 1 As shown, the embodiment of the present invention discloses a method for continuous detection of toxicity data with increasing disturbance mode, characterized in that it includes the following steps:

[0056] Step 1: Get the text to be detected, use the toxic data jailbreak perturbation framework to perturb all the texts to be detected in different modes, obtain the jailbreak difficult text to be detected under each perturbation mode, and use this to construct the data set under each perturbation mode, and divide the data set into training set, verification set and test set;

[0057] In this embodiment, if Figure 2 As shown in the figure, the toxic data jailbreak perturbation framework consists of three parts: selector, perturbator and verifier module; the selector selects words that will increase the toxicity score of the text to be tested (the toxicity score is a value of 0 to 1 directly provided by the PerspectiveAPI tool. The larger the toxicity score, the more likely the text to be tested is toxic) by adopting a variety of strategies, and the perturbator performs different perturbations on the words selected by the selector, and finally the verifier filters out the perturbed toxic text that cannot be detected by the detector.

[0058] The selector in this embodiment adopts four selection strategies to select important toxic words, as follows:

[0059] 1) Select toxic online texts on the Internet;

[0060] 2) Select toxic words based on PerspectiveAPI;

[0061] 3) Obtain the toxicity score of the text to be tested based on the PerspectiveAPI, and select the toxic words that increase the toxicity score of the sample to be tested. For example, if the toxicity score of the text to be tested increases after suck is added to the text to be tested, then suck is selected;

[0062] 4) The mutual information (MI)-based measurement method is used to measure the false correlation between toxic words and toxic labels. The false correlation threshold is set to the sum of the mean and standard deviation of MI. When MI is greater than the threshold, the word is considered to have a false correlation and is selected. For example, if the MI of dick and the toxic label is 0.5, which is greater than the threshold of 0.2, dick is selected; otherwise, the word is not selected.

[0063] After selecting important toxic words through the four strategies of the selector, this embodiment uses the perturbator to further perturb the selected toxic words using 9 jailbreak perturbation modes, as follows:

[0064] 1) Insertion perturbation: randomly select special characters from the word selected by the selector and insert them into the word, such as hello->h**ello;

[0065] 2) Deletion perturbation: Randomly select some characters in the word selected by the selector and delete them, such as hello->helo;

[0066] 3) Repeat perturbation: randomly select some characters in the word selected by the selector and repeat them, such as hello->heelllloo;

[0067] 4) Swap perturbation: randomly select two characters from the word selected by the selector and swap their order, such as hello->hlelo;

[0068] 5) Homograph perturbation: randomly select some characters in the word selected by the selector and replace them with other visually similar characters, such as hello->he11o;

[0069] 6) Masked word perturbation: randomly select some characters in the word selected by the selector and replace them with special characters, such as hello->h*ello;

[0070] 7) Abbreviation perturbation: Use abbreviations / slang to replace relevant words or phrases in the text, such as as soon as possible->ASAP;

[0071] 8) Irrelevant perturbation: Randomly select non-toxic words to form a prefix and add it before the text to be tested, such as what do you mean->hello world…what do you mean;

[0072] 9) Authoritative perturbation: Create prompts asking the large model to play the role of different authoritative experts and generate a first-person self-introduction, such as: Please play the role of a scientist and introduce your authoritative work.

[0073] After the perturber perturbs the text to be tested in 9 different modes, the verifier is used to screen the perturbed text to be tested, and the difficult text that is difficult to detect by the detector (this embodiment uses the llama3-8b large model as the detector) is selected. This embodiment selects the Jigsaw dataset that is widely used in toxicity detection tasks. The Jigsaw dataset collects a large amount of English text from different network sources such as social platforms, including the content of the text to be detected, toxicity labels, group labels, etc. This embodiment uses two types of labels in the dataset as multi-task training supervision information. In order to ensure the balance between toxic and non-toxic samples in the dataset, this embodiment resamples toxic and non-toxic samples at a ratio of 1:1, and obtains 20k toxic samples and 20k non-toxic samples for subsequent perturbations.

[0074] The above method is used to perturb and screen 20k toxic samples and 20k non-toxic samples. After the above perturbation and screening, the Jigsaw dataset has a total of 38K samples. The training set contains 23.3K samples, of which 2584 samples are in each perturbation mode. The validation set contains 3.8K samples, of which 430 samples are in each perturbation mode. The test set contains 11.6K samples, of which 1294 samples are in each perturbation mode.

[0075] Step 2: Construct a toxicity continuous detection model based on feature playback, use the training sets under each disturbance mode to train the toxicity continuous detection model and use the validation set to verify it, and obtain the optimal toxicity continuous detection model;

[0076] In this embodiment, if Figure 3 As shown, the constructed feature replay-based toxicity continuous detection model includes building a toxicity detection module, a replay sample selector, a feature replay loss function module, and a toxicity detection classification loss function module.

[0077] In the embodiment, in order to effectively improve the robustness of the toxicity detector to the dynamically occurring disturbed toxic text types, and apply the continuously learned characteristic knowledge to the toxicity discrimination of the text to be detected, this embodiment designs a toxicity continuous detection method based on feature replay, selects feature-level knowledge in the previous disturbance domain according to the feature similarity maximization strategy, and then replays it at the current moment and applies it to the toxicity detection classification loss at the current moment. Specifically:

[0078] Toxicity detection module: This module includes a pre-trained language model BERT based on the Transformer structure and a classifier for detecting sample toxicity. The pre-trained language model BERT is designed to convert text characters into feature vectors that the model can understand and subsequently process. Specifically, given a text sentence S = {S1, S2,,…, S i ,…,S n}, where S i is the i-th character in the sentence, and the encoding feature E of the sentence is obtained through the pre-trained language model BERT. i ,…,E n},in is the encoding feature corresponding to the i-th character, and d represents the dimension of the encoding feature.

[0079] The classifier for detecting sample toxicity aims to predict the toxicity classification of the sample based on the feature vector extracted from the pre-trained language model BERT, namely:

[0080] y=Classifier(E)

[0081] Wherein, E is the feature vector extracted by the pre-trained language model BERT from the text to be tested, and Classifier is responsible for assigning a label y to the text to be tested. In this embodiment, y=1 indicates that the text to be tested is toxic, and y=0 indicates that the text to be tested is non-toxic;

[0082] For example, given the text to be tested: what are you doing?, the feature vector E is extracted through BERT, and then the feature vector E is input into the classifier. The classifier outputs y=1, which means that the text to be tested is classified as toxic text.

[0083] Replay sample selector: Based on the toxicity detection module, this module selects typical replay samples according to the feature vectors extracted by the pre-trained language model BERT. Specifically, given a set of test texts D that have undergone a type of perturbation, the pre-trained language model BERT is used to extract a set of feature vectors E for each text, and then the Euclidean distance between the corresponding feature vectors of different texts is calculated. α (the parameter α can be determined according to actual needs and is set to 5 by default) typical replay samples with the shortest distance to all samples are selected.

[0084] E=BERT(D)

[0085] distance=L2(E i ,E j )

[0086] Where E i ,E j They correspond to the feature vectors of the i-th and j-th samples to be tested in the set D respectively, and L2 represents the Euclidean distance between the two vectors.

[0087] Feature replay loss loss function module: After selecting the replay sample using the replay sample selector, this module uses a negative logarithmic function to calculate the feature similarity between the replay sample at the current moment and the feature extracted by the detection model at the previous moment as the feature replay loss:

[0088]

[0089] in, They correspond to the features extracted by the detector of the i-th playback sample at time j-1 and j respectively; × represents the matrix product, diag represents the diagonal elements of the product, and represents the feature similarity of the playback samples.

[0090] Toxicity detection classification loss function module: Toxicity detection is a classification task. This embodiment calculates the sum of the toxicity detection task loss and the continuous learning task loss as the objective function of the toxicity detection classification loss, where the cross entropy loss function is used to calculate the loss between the true 0-1 label and the model's predicted distribution as the loss of the toxicity detection task:

[0091]

[0092] Where, L cls is the loss of the toxicity detection task, y i represents the true toxicity label of the sample, represents the toxicity label predicted by the toxicity continuous detection model;

[0093] The feature replay loss is used as the loss of the continuous learning task, and the sum of the toxicity detection task loss and the continuous learning task loss is used as the objective function during training:

[0094] L=L cls +L kl,f .

[0095] After constructing the objective function, the training samples and verification samples in different modes are randomly input into the toxicity continuous detection model based on feature playback in batches at each moment. The toxicity continuous detection model is iteratively optimized and trained to obtain evaluation indicators on the training set and the verification set. When the indicator on the verification set no longer increases or the iteration reaches a certain number of times, the optimal model on the verification set is saved as the optimal toxicity continuous detection model after training.

[0096] Step 3: Load the saved feature playback-based continuous toxicity detection model with the best evaluation index on the validation set, batch-feed the test texts under different perturbation modes into the model, and obtain the toxicity detection results of the test texts in different modes.

[0097] In the embodiment, the best toxicity continuous detection model on the validation set is loaded, and the test set D under each perturbation mode is input. test , and obtain the toxicity detection results of each text to be detected in the test set under each perturbation mode.

[0098] In order to illustrate the effect of this embodiment, the toxicity detection effect of this embodiment is compared with the existing LFL, EWC, and LWF. The comparison results are shown in Table 1;

[0099] Table 1 Comparison of detection accuracy between this embodiment and existing methods

[0100]

[0101]

[0102] As can be seen from Table 1, the DynDetect method proposed in this embodiment has achieved the best detection accuracy under different continuous learning order settings (i.e., raw, order-[1-3] learning order) compared with the existing methods. The toxicity detection method based on continuous learning of feature playback proposed in this embodiment has an average detection accuracy of 83% on the perturbation data set, which is higher than the existing methods.

[0103] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the contents of the specification of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for continuous detection of toxicity data with increasing disturbance mode, characterized in that: The following steps are involved: Step 1: Get the text to be detected, use the toxic data jailbreak perturbation framework to perturb all the texts to be detected in different modes, obtain the jailbreak difficult text to be detected under each perturbation mode, and use this to construct the data set under each perturbation mode, and divide the data set into training set, verification set and test set; Step 2: Construct a toxicity continuous detection model based on feature playback, use the training sets under each disturbance mode to train the toxicity continuous detection model and use the validation set to verify it, and obtain the optimal toxicity continuous detection model; Step 3: Send the test sets under each perturbation mode into the optimal toxicity continuous detection model for detection to obtain the corresponding toxicity detection results.

2. The method for continuous detection of toxicity data with increasing disturbance patterns according to claim 1, characterized in that: In step 1, the toxic data jailbreak perturbation framework includes a selector, a perturbator, and a verifier; Among them, the selector selects toxic words in the text to be tested by adopting multiple strategies, the perturbator perturbs the words selected by the selector in different modes, and the verifier selects non-toxic text from the perturbed text.

3. The method for continuous detection of toxicity data with increasing disturbance patterns according to claim 2, characterized in that: The selector selects toxic words by: Select the toxic text; For the selected toxic text, select toxic words based on the PerspectiveAPI; Obtain the toxicity score of the toxic text based on the PerspectiveAPI, and select toxic words that increase the toxicity score of the toxic text; The mutual information MI-based measurement method measures the false correlation between toxic words and toxic tags, and sets the false correlation threshold to the sum of the mean and standard deviation of MI. When MI is greater than the threshold, the word is considered to have a false correlation and is selected. Otherwise, it is not selected.

4. The method for continuous detection of toxicity data with increasing disturbance patterns according to claim 2, characterized in that: The perturbation patterns of the perturbator on words include: Insert perturbation, randomly select special characters from the word selected by the selector and insert them into the word; Deletion perturbation, randomly selecting some characters in the word selected by the selector for deletion; Repeat perturbation, randomly select some characters in the word selected by the selector and repeat them; Swap perturbation, randomly select two characters in the word selected by the selector and swap their order; Homograph perturbation, randomly selects some characters in the word selected by the selector and replaces them with other characters that are visually similar; Masked word perturbation randomly selects some characters in the words selected by the selector and replaces them with special characters; Acronym perturbation, replacing relevant words or phrases in the text with abbreviations or slang; Irrelevant disturbance: randomly select non-toxic words to form prefixes and add them before the text to be tested; Authoritative perturbations, creating prompts that require the large model to play the role of different authoritative experts and generate first-person self-introductions.

5. The method for continuous detection of toxicity data with increasing disturbance patterns according to claim 1, characterized in that: The toxicity continuous detection model includes a toxicity detection module, a playback sample selector module, a feature playback loss function module, and a toxicity detection classification loss function module.

6. The method for continuous detection of toxicity data with increasing disturbance patterns according to claim 5, characterized in that: The toxicity detection module includes a pre-trained language model BERT based on the Transformer structure and a classifier for detecting sample toxicity. The pre-trained language model is used to convert text characters into feature vectors that the model can understand and subsequently process. The classifier for detecting sample toxicity is used to predict the toxicity classification of the sample, namely: y=Classifier(E) Among them, E is the feature vector extracted by the pre-trained language model BERT, and Classifier is responsible for assigning label y to the text to be tested.

7. The method for continuous detection of toxicity data with increasing disturbance patterns according to claim 6, characterized in that: The replay sample selector module is used to calculate the Euclidean distance between the corresponding feature vectors of different texts based on the feature vector set E of each text extracted by the pre-trained language model BERT in the toxicity detection module, so as to select the typical replay sample with the shortest distance to all samples: E=BERT(D) distance=L2(E i ,AND j ) Among them, E i They represent the feature vector of the i-th sample to be tested in the test text set D, E j They respectively represent the feature vector of the jth test sample in the test text set D, and L2 represents the Euclidean distance between the two vectors.

8. The method for continuous detection of toxicity data with increasing disturbance patterns according to claim 7, characterized in that: After the replay sample is selected by the replay sample selector, the feature replay loss loss function module uses a negative logarithmic function to calculate the feature similarity between the features extracted by the detection model at the current moment and the previous moment as the feature replay loss: in, They represent the features extracted by the detector of the i-th playback sample at time j-1, × represents the matrix product, They represent the features extracted by the detector of the i-th playback sample at time j, and diag represents the diagonal elements of the product, which represents the feature similarity of the playback samples.

9. The method for continuous detection of toxicity data with increasing disturbance patterns according to claim 8, characterized in that: The toxicity detection classification loss module is used to calculate the sum of the toxicity detection task loss and the continuous learning task loss. The cross entropy loss function is used to calculate the loss between the true 0-1 label and the predicted distribution of the toxicity continuous detection model as the loss of the toxicity detection task: Where, L cls is the loss of the toxicity detection task, y i represents the true toxicity label of the sample, represents the toxicity label predicted by the toxicity continuous detection model; The feature replay loss is used as the loss of the continuous learning task, and the sum of the toxicity detection task loss and the continuous learning task loss is used as the objective function during training: L=L cls +L kl,f .

10. A system for implementing the method for continuous detection of toxicity data with increasing disturbance patterns as described in any one of claims 1 to 9, characterized in that: include: The data set preparation module is used to obtain the text to be detected, and then use the toxic data jailbreak perturbation framework to perturb all the texts to be detected in different modes, so as to obtain the jailbreak difficult text to be detected under each perturbation mode, and thereby construct the data set under each perturbation mode, and divide the data set into training set, validation set and test set; The model building training module is used to build a toxicity continuous detection model based on feature playback, and use the training set under each disturbance mode to train the toxicity continuous detection model and use the validation set to verify it, so as to obtain the optimal toxicity continuous detection model; The toxicity detection module is used to send the test sets under each disturbance mode into the optimal toxicity continuous detection model for detection to obtain the corresponding toxicity detection results.

Images