A method for identifying hidden viruses in text based on large models

Abstract

The present invention provides a method for identifying hidden poison in text based on a large model, which relates to the field of Internet security technology. The method includes building a text hidden poison detection model by adding a classification head composed of a fully connected layer and a softmax layer to the top layer of the RoBERTa model; crawling comment text data from social software and performing data cleaning and data preprocessing, and classifying the comment text data into obviously poisonous text, hidden poisonous text or non-toxic text through manual data annotation; inputting the comment text data with manual data annotation into the text hidden poison detection model for supervised learning; the text to be identified is input into the text hidden poison detection model after supervised learning, and the classification result of the identified obvious poisonous text, hidden poisonous text or non-toxic text is output. The method can classify the input text into obviously poisonous text, hidden poisonous text and non-toxic text, that is, realize the effective identification of hidden poisonous text.

Description

Technical Field

[0001] The present invention relates to the field of Internet security technology, and in particular to a method for identifying hidden viruses in text based on a large model. Background Art

[0002] At present, in the field of natural language processing involving Internet security technology, toxicity detection mainly relies on technical means such as manual monitoring, convolutional neural network models, and recurrent neural network models.

[0003] While methods for identifying and addressing overtly toxic speech have matured, identifying covertly toxic speech remains a pressing technical challenge. Furthermore, purifying toxic speech, eliminating its toxicity, and achieving textual detoxification, while maintaining the integrity and accuracy of the underlying toxic speech, remains a pressing technical challenge. Summary of the Invention

[0004] To solve the above technical problems existing in the prior art, the present invention provides a method for identifying hidden viruses in text based on a large model. The specific technical solution includes the following steps:

[0005] Step S1: Build a text hidden virus detection model, specifically by adding a classification head consisting of a fully connected layer and a softmax layer on the top layer of the RoBERTa model;

[0006] Step S2: crawl comment text data from social software and perform data cleaning, data preprocessing and manual data annotation;

[0007] Manual data annotation specifically involves: using classification labels to classify comment text data into explicit toxic text, implicit toxic text, or non-toxic text;

[0008] Step S3: Input the manually annotated comment text data into the text hidden virus detection model for supervised learning;

[0009] Step S4: Input the text to be identified into the supervised learning text hidden virus detection model and output the identification classification result.

[0010] Preferably, data preprocessing includes word segmentation, sentence segmentation and removal of stop words.

[0011] Preferably, before step S3, the method further includes performing data enhancement on the comment text data using the GeniusAug model;

[0012] The GeniusAug model consists of an input layer, a feature extraction layer, a bidirectional encoding layer, a sketch construction layer, an autoregressive decoding layer, and an output layer connected in sequence;

[0013] The data augmentation process is as follows:

[0014] Input layer: input comment text data d and the corresponding first category label;

[0015] Feature extraction layer: extract the n-grams [w1,w2,...,w m ], n = 1, 2, 3; where m is the number of words in the comment text data d;

[0016] Bidirectional encoding layer: Encode the comment text data d, the first category label and the n-gram to obtain the text embedding e of the comment text data d d 、n-grams[w1,w2,...,w m ] is embedded in the syntax [v1,v2,...,v m ], label embedding e of the first category label t ;

[0017] For the review text data d, calculate each n-gram and fusion embedding e f The similarity S i , the formula is as follows:

[0018] e f =λe d +(1-λ)e t ;

[0019]

[0020] In the formula, λ is the preset fusion weight, i is the word ordinal number;

[0021] Select the top A% of n-grams as keywords;

[0022] Sketch construction layer: Based on the keywords, the sketch is constructed by applying the extraction masking projection process;

[0023] Autoregressive decoding layer: Generates new text data based on the sketch using a beam search method using the pre-trained GENIUS model;

[0024] Output layer: outputs new text data.

[0025] Preferably, after constructing the sketch, the method further includes adding a theme or emotional hint to the front end of the sketch.

[0026] Furthermore, the method further comprises the following steps:

[0027] Step C1: Construct an implicit toxicity span model, including an input layer, an embedding layer, a bidirectional LSTM layer, a dense layer, a CRF layer, and an output layer connected in sequence;

[0028] Step C2: Input the hidden toxic text identified by the text hidden toxicity detection model into the trained hidden toxicity span model and output the span label;

[0029] Step C3: Based on the span labels, the seq-to-seq model of the RoBARTa model is used to rewrite or replace the toxic content in the hidden toxic text to generate normalized span labels;

[0030] Step C4: Merge the normalized span labels with the non-toxic content in the hidden toxic text to generate the detoxified text;

[0031] Step C5: Calculate the toxicity intensity φ of the detoxified text t',

[0032] Rt'=τ-φt';

[0033] L = l + (1 - R);

[0034] Where R is the penalty term, τ is the toxicity intensity threshold, L is the comprehensive loss function, and l is the original loss function;

[0035] Step C6: Ask the user whether to publish the detoxified text t';

[0036] If yes, output the detoxified text t', otherwise output the hidden poison text with * replacing the poisonous content;

[0037] The training process of the implicit toxicity span model includes:

[0038] Input layer: For each first hidden poison text classified by manual data annotation, a range label is manually annotated to generate a text label sequence. The range label includes a start label, a continuation label, and an end label.

[0039] Embedding layer: converts the first hidden toxic text and the corresponding text label sequence into a vector representation and inputs it into the implicit toxicity span model;

[0040] Bidirectional LSTM layer: The bidirectional LSTM layer captures the context representation of each first hidden poison text and obtains the hidden state representation of each first hidden poison text.

[0041] Dense layer: flattens the hidden state representation of each first hidden text;

[0042] CRF layer: After determining the text label sequence feature function based on the flattened hidden state representation, it combines the weight parameter w to model the dependency relationship between labels in each range;

[0043] Map the text label sequence feature function to a high-dimensional feature vector and calculate the conditional probability P(s|x;w) of the text label sequence.

[0044]

[0045] Where x represents the first hidden poison text, s represents the text label sequence, and φ(x,s) represents mapping the first hidden poison text x and the text label sequence s into a high-dimensional feature vector;

[0046] Output layer: Adjust the weight parameter w and use the text label sequence with the highest probability as the span label s of the first hidden poison text * ;

[0047] s * =arg max p(s|x;w * );

[0048] Where w * is the optimal weight parameter, p(s|x; w * ) is the optimal conditional probability of the text label sequence.

[0049] In the technical solution provided by the present invention, the trained text hidden poison detection model can classify the input text into obvious poison text, hidden poison text and non-toxic text, that is, realize the effective identification of hidden poison text. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Schematic diagram of the process of identifying hidden viruses in Chinese texts in the present invention.

[0051] Figure 2 Schematic diagram of the process of building an enhanced dataset in the present invention.

[0052] Figure 3 The figure is a flow chart of processing hidden poison text in the present invention. DETAILED DESCRIPTION

[0053] Hereinafter, the technical solution provided by the present invention will be further elaborated in conjunction with the accompanying drawings.

[0054] At present, network text detection mainly relies on technical means such as manual monitoring, convolutional neural network models, and recurrent neural network models. At present, there are good identification methods for explicit toxic language, but the identification of implicit toxic language is still a challenge. In addition, in order to purify the network environment, the field of natural language processing has proposed a text detoxification task, which aims to modify toxic texts to ensure that the toxicity of the text is eliminated while the useful content is intact and the original meaning remains unchanged. In order to solve the problem of the spread of a large amount of toxic information in cyberspace, the present invention proposes a method for identifying hidden toxic texts based on large models and data enhancement. Specifically, by integrating data enhancement and large model technology, toxic texts in online speech, especially implicit toxicity, can be effectively identified, and toxic speech detection can be performed from multiple perspectives to help users avoid sending toxic information in a timely manner.

[0055] Specifically, if Figure 1As shown, the technical solution provided by the present invention mainly includes the following five steps.

[0056] Step 1: Text data collection. Collect text data, perform data cleaning and data preprocessing, and data annotation on the collected text data to obtain a dataset.

[0057] It specifically includes the following parts:

[0058] Crawling comment text data from Sina Weibo with the most likes. Performing data cleaning and preprocessing on the crawled comment data.

[0059] Data cleaning involves removing duplicate and outlier data to improve data quality. Its primary goal is to ensure data accuracy and completeness, eliminating noise and errors. Data preprocessing includes operations such as word segmentation, sentence segmentation, and stop word removal.

[0060] Data cleaning is different from data preprocessing: new data problems may be discovered during data preprocessing, which require returning to the data cleaning stage for processing; at the same time, the results of data cleaning may also affect subsequent data preprocessing steps.

[0061] Manual data annotation is performed on the processed data, and the annotation follows the following rules:

[0062] (1) Toxic text: covers a variety of abusive language phenomena, such as hate language, identity attacks, blasphemy or aggression. For example: So-and-so is uncivilized!

[0063] (2) Hidden toxic text: This type of toxic language may appear harmless on the surface, but it often uses complex language forms such as irony, exaggeration, and rhetorical questions to conceal its true intentions. For example: 1, 3, 4, 5, A, C, D, E, but you are missing. This sentence may seem harmless, but in fact it links the missing "2" and "B" in "1, 3, 4, 5" and "A, C, D, E" with the object "you", making a riddle-like language attack on "you". This inappropriate statement is not only illogical, but also derogatory. This expression implies disrespect and discrimination. Although it does not directly use offensive words, its inherent meaning is derogatory.

[0064] (3) Non-toxic text: The content is generally harmless and does not contain any form of abuse or derogatory remarks. Without context or additional information, it may be difficult to understand, but it is not inherently offensive or abusive. For example: Huahua is so cute!

[0065] Each text is annotated by multiple people, and the principle of minority obeys majority is adopted for data with different annotation results.

[0066] Step 2: Perform data augmentation on the data annotated in step 1 to obtain a new data set.

[0067] Fine-tuning the data-augmented GeniusAug model. The core GeniusAug module consists of a bidirectional encoder and an autoregressive decoder. By using toxicity-aware labels to generate text data similar to the original dataset, we effectively increased the diversity and number of samples in the training set. This enhanced data diversity enables the classification model to better learn and generalize from a wider range of data.

[0068] Among them, the GeniusAug model is as follows Figure 2 As shown, it includes an input layer, a feature extraction layer, a bidirectional encoding layer, a sketch construction layer, an autoregressive decoding layer, and an output layer connected in sequence.

[0069] The data enhancement process is as follows:

[0070] Input layer: Input comment text data d and the corresponding first category label.

[0071] Feature extraction layer: extract the n-grams [w1,w2,...,w m ], n = 1, 2, 3; where m is the number of words in the comment text data d.

[0072] Bidirectional encoding layer: Encode the comment text data d, the first category label and the n-gram to obtain the text embedding e of the comment text data d d 、n-grams[w1,w2,...,w m ] is embedded in the syntax [v1,v2,...,v m ], label embedding e of the first category label t .

[0073] Comment text data d , calculate each n-gram and fusion embedding e f The similarity S i , the formula is as follows:

[0074] e f =λe d +(1-λ)e t ;

[0075]

[0076] Where λ is the preset fusion weight, and i is the word ordinal number.

[0077] The top 20% of n-grams with the highest similarity are selected as keywords. This ratio can be fine-tuned based on actual needs.

[0078] Sketch construction layer: Based on the keywords, sketches are constructed by applying an extraction masking projection process.

[0079] Autoregressive decoding layer: Generates new text data based on the sketch using the beam search method through the pre-trained GENIUS model.

[0080] Output layer: outputs new text data.

[0081] During the data augmentation process, the core semantics and outline of the original text are preserved. The data augmentation pre-training task differs significantly from the previous denoising pre-training objective in two key ways: extreme masking, which masks up to 80% of the text; and selective masking, which, based on the sketch extraction pipeline, selectively masks less informative portions of the text, rather than randomly masking tokens or spans of the original text. After the sketches are completed, new training samples are generated based on these sketches using the pre-trained GENIUS model. Furthermore, the generated text retains key components of the sketches, ensuring that the text does not deviate significantly from the original text semantically. To further control the properties of the generated text, thematic or sentiment cues can be added before the sketches. The ability to generate content tailored to specific attributes is gained from the reconstruction process of the sketch pre-training.

[0082] Step 3: Build a large model (text hidden virus detection model)

[0083] To detect text toxicity, the dataset obtained in step 2 is fine-tuned on the RoBERTa pre-trained model. First, a classification head consisting of a fully connected layer and a softmax layer is added to the top layer of the RoBERTa model to output a probability distribution for each category. The pre-trained RoBERTa model is then subjected to supervised learning using the prepared dataset from step 2. Finally, a self-attention layer with a linear activation of the fully connected layer is used to predict the toxicity type of the input sample. Typically, the category with the highest probability is selected as the final classification result. After outputting the classification results for toxic speech, explicitly toxic speech is directly blocked. For non-toxic comments, the original comments are directly output. The classification results for implicitly toxic speech are further processed.

[0084] like Figure 3 As shown, further processing of hidden poison text includes steps 4 and 5.

[0085] Step 4: Build and train an implicit toxicity span model

[0086] The implicit toxicity span model consists of an input layer, an embedding layer, a bidirectional LSTM layer, a dense layer, a CRF layer, and an output layer, which are connected sequentially.

[0087] The training process of the implicit toxicity span model includes:

[0088] Input layer: For each first hidden poison text classified by manual data annotation, a range label (B, I, O) is manually labeled to generate a text label sequence; among them, the range label includes the start label B, the continuation label I and the end label O.

[0089] Embedding layer: Convert the first hidden toxic text and the corresponding text label sequence into a vector representation and input it into the implicit toxicity span model.

[0090] Bidirectional LSTM layer: The bidirectional LSTM (Long Short-Term Memory) layer captures the contextual representation of each first hidden poison text and obtains the hidden state representation of each first hidden poison text.

[0091] Temporally distributed dense layers: Flatten the hidden state representation of each first hidden context.

[0092] CRF (Conditional Random Field) layer: After determining the feature function of the text label sequence based on the flattened hidden state representation, it combines the weight parameter w to model the dependency relationship between labels in each range.

[0093] Map the text label sequence feature function to a high-dimensional feature vector (d dimension) and calculate the conditional probability P(s|x;w) of the text label sequence.

[0094]

[0095] Where x represents the first hidden poison text, s represents the text label sequence, and φ(x,s) represents mapping the first hidden poison text x and the text label sequence s into a high-dimensional feature vector;

[0096] The range of the text label sequence s spans different sequences. The most likely label for a sentence can be found as follows: The toxic portion of a sentence is the portion responsible for conveying toxicity. If this toxic content can be normalized (i.e., detoxified), sentence-level normalization can be achieved. A single example can have multiple non-overlapping toxic ranges; the goal of implicit toxicity span is to identify all toxic ranges.

[0097] Output layer: Adjust the weight parameter w and use the text label sequence with the highest probability as the span label s of the first hidden poison text * ;

[0098] s * =arg max p(s|x;w * );

[0099] Where w * is the optimal weight parameter, p(s|x; w *) is the optimal conditional probability of the text label sequence.

[0100] Step 5: Build the detoxification module

[0101] The detoxification model is based on a Generative Adversarial Network (GAN) architecture. This model uses steps 3 and 4 as auxiliary modules. The detoxification module accepts toxic samples along with span labels identified by an implicit toxic span model. For the detoxification module, the pre-trained RoBARTa model is used and fine-tuned for the detoxification task.

[0102] According to the span labels, the seq-to-seq model based on the RoBARTa model rewrites or replaces the toxic content in the hidden toxic text to generate normalized span labels;

[0103] Merge the normalized span labels with the non-toxic content in the hidden toxic text to generate the detoxified text;

[0104] The toxicity intensity φ of the detoxified text t' is calculated through the discriminator model of the text hidden poison detection model.

[0105] Rt'=τ-φt';

[0106] L = l + (1 - R);

[0107] Where R is the penalty term, τ is the toxicity intensity threshold, L is the comprehensive loss function, and l is the original loss function;

[0108] Ask the user whether to publish the detoxified text t';

[0109] If so, output the detoxified text t', otherwise output the hidden poison text with * replacing the poisonous content.

[0110] If the toxicity intensity φ of the detoxified text t' is still greater than the preset intensity, it can be repeatedly rewritten or replaced, and the comprehensive loss can be minimized through backpropagation to generate semantically coherent normalized samples with a hidden toxicity intensity less than or equal to 1. The goal of detoxification is to generate new samples that retain the original semantics but have lower toxicity. Intuitively, it is first necessary to determine the threshold between strong and weak toxicity. Set the threshold τ = 5. For each strongly toxic sample φt'>τ, the goal is to generate semantically similar samples with the constraint φt'≤τ.

[0111] It can be seen that in the data enhancement module design of the present invention, the data enhancement method based on the bidirectional encoder and autoregressive decoder is innovative, which can effectively increase the diversity and quantity of review text data and save the cost of manual labeling.

[0112] By fine-tuning and building upon the pre-trained RoBERTa model and the bidirectional LSTM model, we constructed a text implicit toxicity detection model and an implicit toxicity span model, respectively, to predict the toxicity intensity of text and identify toxic spans within it. The innovative approach of fine-tuning pre-trained models and combining models is able to improve the model's performance in both toxicity detection and span identification tasks.

[0113] The detoxification model based on generative adversarial networks is designed to remove the text toxicity module. It accepts toxic samples along the span labels identified by the implicit toxic span model and generates new samples that retain the original semantics but have lower toxicity. Among them, the detoxification model design based on GAN is innovative and can effectively generate non-toxic text that meets semantic requirements.

[0114] In summary, it can be seen that in the technical solution provided by the present invention, the trained text hidden poison detection model can classify the input text into obvious poison text, hidden poison text and non-toxic text, that is, realize the effective identification of hidden poison text.

[0115] Furthermore, on the basis of achieving the above beneficial effects, each preferred scheme also achieves the following beneficial effects: data preprocessing is conducive to improving the accuracy of subsequent data standards; data enhancement can effectively increase the diversity and quantity of comment text data while saving manual annotation costs; generating span tags and detoxifying based on span tags can realize the detoxification of hidden toxic texts and generate non-toxic texts that meet semantic requirements.

Claims

1. A method for identifying hidden viruses in text based on a large model, characterized by: The following steps are involved: Step S1; Build a text hidden poison detection model by adding a classification head consisting of a fully connected layer and a softmax layer on top of the RoBERTa model; Step S2: crawl comment text data from social software and perform data cleaning, data preprocessing and manual data annotation; Manual data annotation specifically involves: using classification labels to classify comment text data into explicit toxic text, implicit toxic text, or non-toxic text; Step S3: Input the manually annotated comment text data into the text hidden virus detection model for supervised learning; Step S4: Input the text to be identified into the supervised learning text hidden virus detection model, and output the identification classification result; Among them, the classification results include obvious toxic text, hidden toxic text and non-toxic text; Before step S3, the review text data is also enhanced using the GeniusAug model; The GeniusAug model consists of an input layer, a feature extraction layer, a bidirectional encoding layer, a sketch construction layer, an autoregressive decoding layer, and an output layer connected in sequence; The data augmentation process is as follows: Input layer: input comment text data d and the corresponding first category label; Feature extraction layer: extract the n-grams [w1,w2,...,w m ], n = 1, 2, 3; where m is the number of words in the comment text data d; Bidirectional encoding layer: Encode the comment text data d, the first category label and the n-gram to obtain the text embedding e of the comment text data d d 、n-grams[w1,w2,...,w m ] is embedded in the syntax [v1,v2,...,v m ], label embedding e of the first category label t ; For the review text data d, calculate each n-gram and fusion embedding e f The similarity S i , the formula is as follows: e f =λe d +(1-λ)e t ; In the formula, λ is the preset fusion weight, i is the word ordinal number; Select the top A% of n-grams as keywords; Sketch construction layer: Based on the keywords, the sketch is constructed by applying the extraction masking projection process; Autoregressive decoding layer: Generates new text data based on the sketch using a beam search method using the pre-trained GENIUS model; Output layer: outputs new text data.

2. A method for identifying hidden viruses in text based on a large model as claimed in claim 1, characterized in that: The data preprocessing includes word segmentation, sentence segmentation and removal of stop words.

3. The method for identifying hidden viruses in text based on a large model as claimed in claim 1, characterized in that: After constructing the sketch, it also includes adding a theme or emotional hint at the front of the sketch.

4. The method for identifying hidden viruses in text based on a large model as claimed in claim 1, characterized in that: The following steps are also included: Step C1: Construct an implicit toxicity span model, including an input layer, an embedding layer, a bidirectional LSTM layer, a dense layer, a CRF layer, and an output layer connected in sequence; Step C2: Input the hidden toxic text identified by the text hidden toxicity detection model into the trained hidden toxicity span model and output the span label; Step C3: Based on the span labels, the seq-to-seq model of the RoBARTa model is used to rewrite or replace the toxic content in the hidden toxic text to generate normalized span labels; Step C4: Merge the normalized span labels with the non-toxic content in the hidden toxic text to generate the detoxified text; Step C5: Calculate the toxicity intensity φ of the detoxified text t', Rt'=τ-φt'; L = l + (1-R); Where R is the penalty term, τ is the toxicity intensity threshold, L is the comprehensive loss function, and l is the original loss function; Step C6: Ask the user whether to publish the detoxified text t'; If yes, output the detoxified text t', otherwise output the hidden poison text with * replacing the poisonous content; The training process of the implicit toxicity span model includes: Input layer: For each first hidden poison text classified by manual data annotation, a range label is manually annotated to generate a text label sequence. The range label includes a start label, a continuation label, and an end label. Embedding layer: converts the first hidden toxic text and the corresponding text label sequence into a vector representation and inputs it into the implicit toxicity span model; Bidirectional LSTM layer: The bidirectional LSTM layer captures the context representation of each first hidden poison text and obtains the hidden state representation of each first hidden poison text. Dense layer: flattens the hidden state representation of each first hidden text; CRF layer: After determining the text label sequence feature function based on the flattened hidden state representation, it combines the weight parameter w to model the dependency relationship between labels in each range; Map the text label sequence feature function to a high-dimensional feature vector and calculate the conditional probability P(s|x;w) of the text label sequence. Where x represents the first hidden poison text, s represents the text label sequence, and φ(x,s) represents mapping the first hidden poison text x and the text label sequence s into a high-dimensional feature vector; Output layer: Adjust the weight parameter w and use the text label sequence with the highest probability as the span label s of the first hidden poison text * ; s * =argmaxp(s|x;w * ); Where w * is the optimal weight parameter, p(s|x; w * ) is the optimal conditional probability of the text label sequence.

Images