A social network sensitive euphemism detection method based on supervised learning

Abstract

The application discloses a social network sensitive euphemism detection method based on supervised learning, first, data on a certain social network platform is collected, including user comments and video bullet screen information, then sensitive euphemism is used for data filtering and strict manual annotation of the data set, a labeled data set sensitive euphemism data set is established, finally, a ConSERT framework based on contrast learning is used for sentence representation, and a sensitive euphemism classification detection model is proposed by combining Bi-LSTM and an attention mechanism, so that effective detection of sensitive euphemism is realized. Experimental evaluation results show that the effect of the SED model on the sensitive euphemism detection problem is better than that of a commonly used sensitive content detection method. The research work of the application provides a method and a train of thought for future sensitive euphemism detection for social networks.

Description

Technical Field

[0001] This invention relates to the field of network security technology in computer science and technology, specifically to a method for detecting sensitive euphemisms on social networks based on supervised learning. Background Technology

[0002] With the continuous development of information technology and mobile internet, social networks have had an increasingly significant impact on people's lives. Social networks have become a crucial force driving the integration of real life and virtual space. Internet users are also increasingly enthusiastic about using social networks for communication and information sharing. In recent years, video sharing platforms on social networks, which have become very popular in China, have provided user-generated content platforms. Registered users can share original videos or forward videos through their personal accounts, offering great freedom and innovation. However, due to the large number of video comments and bullet comments, the large overall user base, and the relatively young age of the users, content creation is prone to include harmful content. Harmful content is found in the comments and bullet comments of some videos on social network video sharing platforms, which is detrimental to the healthy growth of teenagers and also negatively impacts internet content security in my country. Although sensitive content with clearly harmful connotations is easily filtered by existing methods, sensitive euphemisms—language that uses euphemisms to express sensitive content—still exist extensively on public internet platforms. Euphemisms are expressions used to weaken or conceal sensitive, unpleasant, or taboo topics that are subject to the norms of speech in real-world and internet society. The present invention defines sensitive euphemisms as: using the harmless meaning of common words as a disguise, and expressing various sensitive content through malicious misinterpretation (through different euphemisms such as extension, metonymy, and homophones). The words used for disguise are called sensitive euphemisms.

[0003] Because sensitive euphemisms often employ disguised vocabulary within a large number of normal expressions, classic content moderation mechanisms based on sensitive word detection struggle to detect such content. Specifically, on one hand, content moderators find it difficult to identify these sensitive euphemisms because they are frequently embedded in sentences with neutral meanings, subtly expressing sensitive content through euphemisms, thus easily evading moderation. On the other hand, practically speaking, social network platform administrators cannot simply block sensitive euphemisms, as this would result in the prohibition of a large number of normal words, severely impacting users' normal expression. After being marked as sensitive and blocked, ordinary users cannot use the normal meanings of sensitive euphemisms, hindering normal communication; obviously, a larger proportion of users use the harmless meanings of sensitive euphemisms rather than the sensitive meanings used to attack other users. Furthermore, even if some sensitive euphemisms are blocked, malicious attackers can still use similar malicious misinterpretations (through extensions, metonymy, homophones, and other euphemistic methods) to create new sensitive euphemisms, rendering word blocking ineffective. Existing methods use sensitive word list matching to detect sensitive words and enrich the sensitive word database by adding newly identified variant words, but they ignore the correlation between variant words and original words. For the detection of sensitive euphemisms, sensitive word matching cannot effectively solve the real problem of the proliferation of sensitive euphemisms. Furthermore, existing text detection methods do not fully consider sentiment information and semantic features, and suffer from a high number of false positives and low accuracy. Summary of the Invention

[0004] To address the aforementioned problems, the purpose of this invention is to provide a supervised learning-based method for detecting sensitive euphemisms on social networks. This method achieves more accurate and precise detection of sensitive euphemisms, providing a method and approach for future sensitive euphemism detection on social networks. The specific technical solution is as follows:

[0005] A supervised learning-based method for detecting sensitive euphemisms on social networks includes the following steps:

[0006] Step 1: Data collection and annotation: Web crawlers are used to collect data from the social network platforms to be detected, and the collected data is filtered and selected, and then manually annotated to build a sensitive euphemism dataset;

[0007] Step 2: Feature extraction: After initializing the embedding of the input sentence, an initial sentence vector is generated. Different augmented versions of the sentence vector are generated through different data augmentation methods. Then, BERT shared encoding is used to share the sentence vector generation parameters. Then, a contrastive learning layer is used for optimization and adjustment to make the spatial feature distribution of the sentence vector uniform. Finally, the augmented sentence vector is output.

[0008] Step 3: Detection Model: Based on the enhanced sentence vector constructed by the ConSERT model, and combined with Bi-LSTM and attention mechanism, a sensitive euphemism classification and detection model is constructed. The enhanced sentence vector is input into the sensitive euphemism classification and detection model, and sensitive euphemisms on social network platforms are detected based on sentence feature vectors.

[0009] Furthermore, step 1 includes:

[0010] Step 1.1: Design a web crawler based on the Scrapy framework that supports multi-process operation for targeted collection of video comments and bullet comments on the social network platform to be detected;

[0011] Step 1.2: Design a web crawler based on the Scrapy framework that supports multi-process operation, for targeted collection of video comments and bullet comments on a social network platform;

[0012] Step 1.3: Define harmless everyday words that express sensitive content through malicious misinterpretation as sensitive euphemisms. Based on the content and context of the sentence, and referring to the existing sensitive euphemism database, manually label sensitive euphemism sentences that may express similar meanings to specific sensitive words.

[0013] Furthermore, the principles for manually annotating sensitive and euphemistic sentences include:

[0014] 1) If a sensitive euphemism can be reasonably interpreted according to its harmless meaning, but cannot be reasonably interpreted according to its sensitive meaning, then the sentence is considered to contain only harmless meaning, is marked as a harmless sentence, and the tag is set to 0;

[0015] 2) If a sensitive euphemism cannot be reasonably interpreted according to its harmless meaning, but can be reasonably interpreted according to its sensitive meaning, then the sentence is considered to contain only the sensitive meaning, is marked as a sensitive euphemism sentence, and the label is set to 1;

[0016] 3) If the semantic information of a sentence only contains auxiliary words and sensitive euphemisms, and can be interpreted by both harmless and sensitive meanings, then the sentence is considered to contain both sensitive and harmless meanings, and is marked as a sensitive euphemism sentence with a label set to 1.

[0017] 4) If a sensitive euphemism cannot be reasonably interpreted according to its harmless meaning, and also cannot be reasonably interpreted according to its sensitive meaning, then the sentence is considered semantically unclear, filtered out, and not tagged.

[0018] Furthermore, step 1 also includes evaluating the effectiveness of manual annotation, specifically:

[0019] Two annotators independently browse all the sentences to be annotated in the dataset, and calculate... Coefficients are used to evaluate the consistency of the annotation results:

[0020] (1);

[0021] in, , ; The set of sentences annotated by the first annotator. The set of sentences annotated by the second annotator; This is a set of sentences that the first annotator could not determine whether they were sensitive euphemisms. This is a set of sentences for which the second annotator could not determine whether they are sensitive euphemisms. It is the collection of all sentences. It is the size of a set.

[0022] Furthermore, step 2 specifically includes:

[0023] Step 2.1: Input the preprocessed data text through the input layer. ;

[0024] Preprocessed sentences Include One word, The maximum sequence length set will exceed [the limit]. Discard any part of the character; add zeros if necessary.

[0025] Step 2.2: Generate initial sentence vectors by processing the input layer's text data using the optimized BERT model;

[0026] Step 2.2.1: Pre-train the BERT-Base-Chinese model using video comments and bullet screen data from the aforementioned social network platform, and fine-tune the model using the SES-Dataset dataset to obtain the optimized BERT model;

[0027] Step 2.2.2: Transfer the preprocessed data text After tokenization, the tokenized text is obtained. The input is then fed into the optimized BERT model, where word embeddings are extracted and average pooling is applied to obtain high-quality semantic features in the form of sentence embeddings. ; as shown in the following formula:

[0028] (2);

[0029] (3);

[0030] (4);

[0031] Where S is the sentence vector;

[0032] Step 2.3: Generate sample pairs for contrastive learning using different data augmentation strategies through the data augmentation layer;

[0033] Step 2.4: Complete the contrastive prediction task using the contrastive loss layer.

[0034] Given a set Including a pair of positive samples and The purpose of the contrast prediction task is to predict a given... ,exist Identified in ;

[0035] Two separate data augmentation operators are sampled from the same augmentation set and applied to each data sample to obtain two related sample pairs; using the obtained sample pairs, a basic encoder network is then applied. and a projection neural network Conduct training, and remove the projector head after training is complete. and the encoder and represents Used for downstream tasks;

[0036] By random selection A batch of samples is used, and for small batches of augmented sample pairs, a contrastive prediction task is defined to obtain... Given one data point; then, given a positive sample pair, the other data points in a small batch... Augmented samples are treated as negative samples;

[0037] This yields positive sample pairs. The loss function, namely the regularized temperature-scaled cross-entropy loss function, is defined as follows:

[0038] (5);

[0039] in, 1 [ k ≠ a ] ∈ { 0 , 1 } It is if and only if An index function that takes a value of 1 at a given time. Represents temperature parameter; , indicating through After regularization and cosine similarity, , and Samples ,sample and samples The vector.

[0040] Furthermore, the data augmentation strategy in step 2.3 includes adversarial attacks, specifically as follows:

[0041] set up It is a parameter of a model. It is the input of the model, y is the... Related goals, It is the loss function used to train neural networks; the loss function is wrapped around... The current value is linearized as follows:

[0042] (6);

[0043] In the formula, To improve the loss value, This is the learning rate or learning proportion parameter. For gradient calculation;

[0044] The required gradients are calculated using backpropagation to train a single model. To identify labels , If it is the sigmoid function, then the training process with gradient descent is as follows:

[0045] (7);

[0046] in, This is a symbolic representation of the gradient descent training process for the target data. It is the Softplus function; These are the weighting coefficients. This is the bias coefficient;

[0047] The sign of the gradient is just... ,and , Let be the weight vector; then the adversarial version of logistic regression is to minimize the following equation:

[0048] (8);

[0049] in, This is the inner product of the weight vectors when the prediction probability is 1.

[0050] Furthermore, the data augmentation strategy in step 2.3 includes position shuffling, as detailed below:

[0051] The input sentence is shuffled by manipulating its position and sentence embedding. After encoding the shuffled input using BERT, the context embedding is considered. C = [ h c l s , h 0 , . . . , h N − 1 , h s e p ] This corresponds to the insertion into the input sequence. Each sentence token, and the first one [ C L S ] Token and the last one [ S E P ] The token is passed to the sequence reconstructor to reconstruct the original sentence order.

[0052] Except for the last one [ S E P ] Token outside, The embeddings in the code are processed sequentially by the Transformer decoder to obtain the output. Simultaneously consider the entire And the embeddings processed in the previous steps Each processed output will be used in the prediction step. Sentences in As shown in the following formula:

[0053] (9);

[0054] In the formula, For the sentence embedding of the i-th sentence;

[0055] The probability distribution on the sentence representation is computed using a pointer network, where each probability represents the likelihood of each sentence appearing in the first step, as shown in the following formula:

[0056] (10);

[0057] Calculate a cross-entropy loss to match sentences for each context, where the probability distribution of the embedded predicted order is related to the golden position; It is a sentence Appearing in location The predicted probability, Steps A label vector for the correct location in the middle; finally, the loss of all locations is averaged as As shown in the following formula:

[0058] (11);

[0059] In the formula, For the average cross-entropy loss, The sentence position number, This is the label vector for the corresponding position in the sentence. For sentences Appearing in location The predicted probability;

[0060] The model uses only sentence representations to find the original sequence, forcing the encoder to embed all necessary information into the context sentence embeddings instead of expanding them into all embeddings, as shown in the following equation:

[0061] (12);

[0062] In the formula, For the modified cross-entropy loss, This represents the minimum cross-entropy loss.

[0063] Furthermore, the data augmentation strategies in step 2.3 include Cutoff and Dropout;

[0064] Cutoff can be implemented in two ways: using Token Cutoff, randomly selecting a Token and setting the entire row of the corresponding Token's Embedding to zero, or using Feature Cutoff, randomly selecting a Feature from the Embedding and setting the entire column of the selected Feature dimension to zero.

[0065] Dropout specifically involves randomly placing elements in the tag embedding layer with a specific probability, setting their values ​​to zero, and considering each element individually.

[0066] Furthermore, step 3 specifically includes:

[0067] Step 3.1: Extract the sentence vectors The input is fed into a Bi-LSTM network, which leverages its memory function to further learn contextual features, thereby obtaining a one-dimensional semantic feature vector. ; The number of hidden nodes in the network. The calculation method is shown in the following formula:

[0068] (13);

[0069] (14);

[0070] (15);

[0071] in, It is the hidden vector generated by the previous state of the LSTM. It is the hidden vector generated by the next state of the LSTM. Word vectors;

[0072] Step 3.2: The sentence vectors processed by the Bi-LSTM layer are fed into the attention mechanism. Specific sensitive euphemisms are assigned higher weights to improve the detection performance of sensitive euphemisms, as shown in the following formula:

[0073] (16);

[0074] (17);

[0075] (18);

[0076] in, and Represents the weight matrix and bias terms. For the output vector The intermediate hidden layer vector; It is a randomly initialized vector that has already been optimized as model parameters during training; weights Implemented by the Softmax function, it calculates the intermediate vector. and The similarity is used to determine the output vector. weight ;also, The vector used to determine whether a sentence is a sensitive euphemism is used; t represents the top layer. This represents the scaling factor, which is the hyperbolic tangent function;

[0077] Step 3.2: Reduce the dimensionality of the sentence vectors trained by the Bi-LSTM network and attention mechanism, and feed them into the Sigmoid function to obtain the probability that the sentence vector is a sensitive euphemism. This yields the final classification result, as shown in the following formula:

[0078] (19);

[0079] in, This represents the probability that the sentence is a sensitive euphemism.

[0080] Furthermore, the optimization objective of the sensitive euphemism classification and detection model is to minimize the cross-entropy loss function, as shown in the following equation:

[0081] L = − ∑ d ∈ D [ y d l o g p d + ( 1 − y d ) l o g ( 1 − p d ) ] (20);

[0082] in, Represents the sample dataset, Indicates a sample, Represents the true value of the sample. For the sample The probability of predicting a positive class. Minimize the cross-entropy loss for the objective.

[0083] The beneficial effects of this invention are as follows: This invention represents sentences based on the ConSERT framework of contrastive learning, and proposes the SED (Sensitive-Euphemism-Detection) model by combining Bi-LSTM (Bidirectional Long Short-Term Memory) and attention mechanisms, which achieves effective detection of sensitive euphemisms; experimental evaluation results show that the SED model outperforms commonly used sensitive content detection methods in the sensitive euphemism detection problem; the research work of this invention provides methods and ideas for future sensitive euphemism detection for social networks. Attached Figure Description

[0084] Figure 1 This is the overall framework of the supervised learning-based social network sensitive euphemism detection method of the present invention.

[0085] Figure 2 Three different Cutoff data augmentation strategies.

[0086] Figure 3 This is the SED model structure of the present invention.

[0087] Figure 4 This is a comparison chart of the performance of different detection models.

[0088] Figure 5 A comparison chart showing the performance of different sentence embedding models.

[0089] Figure 6 This is a comparison chart showing the detection performance of different classification networks. Detailed Implementation

[0090] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0091] The overall framework of this invention mainly comprises three parts: data collection and annotation, feature extraction, and detection model, such as... Figure 1 As shown.

[0092] (1) Data collection and annotation: Data was obtained by developing a web crawler. Targeted collection was carried out based on seed videos and breadth-first traversal algorithm. Data selection and manual annotation were completed. The sensitive euphemism dataset constructed by the crawler provided data support for this invention.

[0093] (2) Feature extraction: The core objective is to generate accurate and spatially uniform sentence vector representations of sensitive euphemisms. After initializing the embedding of the input sentence, different augmented versions are generated through several different data augmentation methods, such as perturbation and shuffling of Token and PositionIds. This part uses BERT shared encoding to share sentence vector generation parameters, and then uses a contrastive learning layer for optimization and adjustment to make the spatial feature distribution of the sentence vector more uniform, and finally outputs the sentence representation.

[0094] (3) Detection Model: The sentence feature vectors generated in the "feature extraction" module are input into the proposed sensitive euphemism classification and detection model. This model utilizes Bi-LSTM and attention mechanism, which can efficiently detect sensitive euphemisms on social network platforms based on sentence vectors.

[0095] 1. Data Collection and Labeling

[0096] Currently, there is a lack of research on sensitive euphemism detection on certain social networking platforms, resulting in a lack of reliable sensitive euphemism datasets. To address this issue, this invention developed a web crawler based on the Scrapy framework. Using specific strategies, it selectively collected video comments and bullet comments on different topics on the social networking platform, categorized the topics of the videos to which the comment data belonged, and collected a text corpus of nearly 8 million records. The collected data was then filtered and selected, and manually labeled, thus constructing the SES-Dataset (Sensitive-Euphemism-Sentence-Dataset) of this invention. The data collected in this invention includes video information, video comments, and user information.

[0097] 1.1. Data Collection

[0098] (1) Data collection methods

[0099] This invention designs an efficient web crawler that supports multi-process operation to collect various data, which consists of the following four parts: proxy pool, requester, processor, and database.

[0100] Proxy Pool: This social networking platform limits the access frequency of a single IP address. To achieve efficient data acquisition, this invention uses a proxy pool to improve concurrency and crawler efficiency.

[0101] Requester: The requester's tasks are divided into requesting Guest Tokens and requesting data from the social network platform.

[0102] Processor: The processor is mainly used to preprocess and filter the crawled raw data, and then store the processed data in the database.

[0103] Database: The database is divided into video bullet screen database and video comment database.

[0104] (2) Data collection strategy

[0105] Regarding the selection of seed videos, this invention utilizes the channel partitioning of the social network platform, selecting one popular video with a high number of interactions from each of the eight channels with significant topic differences, and using it as the seed video to begin breadth-first traversal. The video selection and filtering are performed in conjunction with the selection principle based on the number of video interactions, and the process is repeated for the next traversal.

[0106] To improve the accuracy of data annotation, this invention establishes several principles for data annotation and evaluates the consistency of annotation results according to the principle of data annotation independence.

[0107] 1.2. Data Labeling

[0108] Definition of sensitive euphemisms: Sentences that express sensitive content (such as vulgarity) through the malicious misinterpretation of harmless everyday words (using methods such as extension, metonymy, and homophones). The harmless everyday words that are maliciously misinterpreted are called sensitive euphemisms. Humans can infer whether a sentence is a sensitive euphemism by examining its content and context.

[0109] Based on the analysis of actual samples, this invention summarizes the characteristics of sensitive euphemisms into the following four types:

[0110] 1) The actual meaning of the sentence is inconsistent with its literal meaning.

[0111] 2) The sentence may contain uncommon phrases or combinations of special letters and numbers.

[0112] 3) The tone of the sentence may have changed significantly compared to the context.

[0113] 4) The meaning of the sentence is inconsistent with the context.

[0114] This invention classifies the data samples obtained through web scraping, combined with video-related tags, and then determines the actual discussion content of sensitive euphemisms into the following five types through manual analysis of topic types:

[0115] 1) Vulgar remarks – Vulgarity type

[0116] 2) Hateful, discriminatory, or offensive remarks against others or groups – Types of Attack

[0117] 3) Criminal speech – a type of crime

[0118] 4) Other categories

[0119] The sensitive euphemisms detected by this invention mainly include vulgar and offensive types.

[0120] Annotation Method: Based on a statistical list of sensitive words (expanding the original sensitive word list with new vulgar and other thematic terms), the platform aims to guide users to correctly use the harmless meanings of certain sensitive euphemisms, ensuring accurate and harmless expression and avoiding ambiguity and misunderstanding. This is the platform's primary goal in content moderation. Annotators review the sensitive word list and annotate sensitive euphemisms that may express similar meanings to certain sensitive words. Specific principles are as follows:

[0121] 1) If a sensitive euphemism can be reasonably interpreted according to its harmless meaning, but cannot be reasonably interpreted according to its sensitive meaning, then the sentence is considered to contain only harmless meaning, is marked as a harmless sentence, and the tag is set to 0;

[0122] 2) If a sensitive euphemism cannot be reasonably interpreted according to its harmless meaning, but can be reasonably interpreted according to its sensitive meaning, then the sentence is considered to contain only the sensitive meaning, is marked as a sensitive euphemism sentence, and the label is set to 1;

[0123] 3) If the semantic information of a sentence only contains auxiliary words and sensitive euphemisms, and can be interpreted by both harmless and sensitive meanings, then the sentence is considered to contain both sensitive and harmless meanings, and is marked as a sensitive euphemism sentence with a label set to 1.

[0124] 4) If a sensitive euphemism cannot be reasonably interpreted according to its harmless meaning, and also cannot be reasonably interpreted according to its sensitive meaning, then the sentence is considered semantically unclear, filtered out, and not tagged.

[0125] Annotation symbols:

[0126] 0 indicates that the sentence is a harmless general sentence (with no harmful meaning), and 1 indicates that the sentence is a sensitive euphemism (with harmful meaning).

[0127] Harmless meanings do not include sensitive words and express healthy, harmonious, and stable content. Harmful meanings may include sensitive words and express content involving vulgar or other unhealthy or negative elements.

[0128] (1) Evaluation of annotation effectiveness

[0129] In terms of data annotation, this invention requires two annotators to independently browse all sentences to be annotated in the dataset and eliminate meaningless text with a large number of repeated words, thereby ensuring the reliability of the data annotation results. Ultimately, this invention calculates... Coefficients are used to evaluate the consistency of the annotation results. The coefficients are defined as shown in equation (1).

[0130] (1);

[0131] in, , . The set of sentences annotated by the first annotator. The set of sentences annotated by the second annotator. This is a set of sentences that the first annotator could not determine whether they were sensitive euphemisms. This is a set of sentences for which the second annotator could not determine whether they are sensitive euphemisms. It is the collection of all sentences. It is the size of a set. In the data annotation process of this invention, The coefficient reached 0.81, indicating that the annotators reached a relatively high consensus on identifying sensitive euphemisms, thus ensuring the reliability of the data annotation results.

[0132] Finally, this invention constructs the SES-Dataset dataset, and Table 1 shows the relevant information of the dataset.

[0133] Table 1 SES-Dataset Information

[0134]

[0135] 2. Feature Extraction

[0136] 2.1. Input Layer

[0137] The input to the SED model proposed in this invention is a labeled text sequence that has undergone data cleaning and selection. For example... Figure 1 As shown, the first part of the input layer will contain preprocessed data text (video comments and video bullet comments). As input. Data preprocessing includes removing web links, foreign characters, special symbols, etc. Preprocessed sentences. Include One word, The maximum sequence length is set ( ), will exceed Words are discarded, and zeros are added if necessary. Since BERT's core unit—the multi-head self-attention mechanism—requires dot products and matrix multiplications between every two symbols in the sequence, longer sequences lead to slow processing speeds and high memory requirements. Considering the experimental environment and the actual dataset, this invention ultimately sets the maximum sequence length to 128.

[0138] 2.2. BERT Coding Layer

[0139] In the BERT encoding layer, this invention uses a fine-tuned BERT model to generate sentence vectors from data-augmented samples. The BERT pre-trained model has achieved outstanding performance in various natural language processing tasks, and its introduction has greatly promoted the development of the field. This BERT encoding layer uses the BERT-Base-Chinese model released by Google, and further pre-trains it using extensive comment and bullet screen data collected from a social network platform, enabling it to learn more features of online language. Based on this, the model is fine-tuned using the SES-Dataset dataset to make it more suitable for the research context of this invention. Currently, among various language detection algorithms based on deep learning, extracting semantic features from text typically uses classic word embedding models such as Word2Vec. Compared to these models, BERT's advantage lies in the context-dependent word vectors it generates. Context-dependent word embeddings can capture other forms of information, producing more accurate feature representations, thereby improving the performance of sensitive euphemism detection models.

[0140] Sentence vectors are vectorized representations of the original sentence sequence, while sentence embeddings are generally used to describe the process of generating sentence vectors.

[0141] Specifically, pre-processed comment or bullet screen text After being tokenized, The words are then fed into the optimized BERT model for word embedding extraction, followed by average pooling to obtain high-quality semantic features in the form of sentence embeddings. , All are 768-dimensional vectors, as shown in equations (2), (3) and (4).

[0142] (2);

[0143] (3);

[0144] (4);

[0145] Where S is the sentence vector.

[0146] 2.3. Data Enhancement Layer

[0147] In the data augmentation layer, this invention uses four different data augmentation strategies to generate samples for contrastive learning.

[0148] (1) Counterattack:

[0149] set up It is a parameter of a model. It is the input of the model, y is the... The associated objective (for machine learning tasks with a specific objective). This is the loss function used to train neural networks. The loss function can be wrapped around... The current value of is linearized to obtain an optimal equation (5), which is called the "fast gradient sign method" for generating adversarial examples. The required gradient can be efficiently calculated using backpropagation.

[0150] (5);

[0151] In the formula, To improve the loss value, This is the learning rate or learning proportion parameter. For gradient calculation.

[0152] To train a single model To identify labels For example, among them If it's the sigmoid function, then the training process with gradient descent includes equation 6:

[0153] (6);

[0154] in, Based on gradient signal perturbation, a simple analytical form is derived to train the worst-case adversarial perturbation. Instead itself. This is a symbolic representation of the gradient descent training process for the target data. These are the weighting coefficients. This is the bias coefficient.

[0155] The sign of the gradient is just... ,and Therefore, the adversarial version of logistic regression aims to minimize equation (7):

[0156] (7);

[0157] in, This is the inner product of the weight vectors when the prediction probability is 1.

[0158] (2) Disorder the positions:

[0159] The original order of the input sentences is shuffled by manipulating their positions and sentence embeddings (because, unlike traditional RNNs, positional encoding is the only clue in the transformer architecture to derive the actual sequence). For example, the input is a pair of swapped sentences, with the positional embeddings of the first sentence being 5 to 7 and the second sentence having positional embeddings of 1 to 4. The sentence embeddings are also swapped in a similar manner. Note that, importantly, the model can perform the shuffling task based solely on the semantics of the sentences and the relationships between them, rather than using positional embeddings, since these are shuffled along with the sentences (i.e., if sentences X and Y are swapped, their positional embeddings are also swapped). For each iteration, only half of the batch is shuffled to allow the model to see some of the input in its natural order.

[0160] After encoding the shuffled input using BERT, consider the embedding of the context: C = [ h c l s , h 0 , . . . , h N t o k e n − 1 , h s e p ] They correspond to those inserted into the input sequence. Each sentence token, and the first one ( [ C L S ] ) and the last one ( [ S E P ] ) The token is passed to the sequence reconstructor to reconstruct the original sentence order.

[0161] Except for the last one outside, The embeddings in the code are processed sequentially by the Transformer decoder to obtain the output. At the same time, pay attention to the whole And the embeddings processed in the previous steps, Each processed output will be used to predict the th... Sentences in Step Because it was inserted earlier , The sentence in the text indicates that the index has been moved. As shown in equation (8):

[0162] (8);

[0163] In the formula, The sentence embedding for the i-th sentence.

[0164] Then, a pointer network is used to compute the probability distribution over the sentence representations. Each probability represents the likelihood of each sentence appearing in the first step, with the aim of predicting their original order, as shown in Equation (9):

[0165] (9);

[0166] A cross-entropy loss is computed to match sentences for each context, where the probability distribution of the predicted order is embedded with the golden position. It is a sentence (because The movement appears in the position The predicted probability, Steps A label vector at the correct location. Finally, the loss at all locations is averaged as... As shown in equation (10):

[0167] (10);

[0168] In the formula, For the average cross-entropy loss, The sentence position number, This is the label vector for the corresponding position in the sentence. For sentences Appearing in location The predicted probability.

[0169] Intuitively, the decoder learns how to transform the representation of one sentence into the representation of the next sentence by focusing on the encoder and the entire sentence representation of the previously transformed embeddings. An important design choice here is that the sequence reconstructor only sees... ,This is ( [ C L S ] ) Sentence markers and ( [ S E P ] ) The contextualized embeddings are masked, while other contextualized word representations are masked. The model should only use sentence representations to find the original sequence, which will force the encoder to embed all the necessary information into the contextualized sentence embeddings, rather than expanding them into all embeddings, as shown in Equation (11):

[0170] (11)

[0171] In the formula, For the modified cross-entropy loss, This represents the minimum cross-entropy loss.

[0172] (3) Cutoff:

[0173] Given a text sequence X = [ x 1 , x 2 , . . . , x L ] Its input embedding matrix is ​​used This indicates. Note that... Indicates the first The embedding vector corresponding to the th tag is the th _ ... dimensionality, This refers to the dimension of the input embedding. Assuming that partial samples can be obtained by cutting the vector along arbitrary dimensions, the proposed method is called Cutoff. Cutoff removes information from the input embedding matrix in a more structured way, by setting a column or row vector to 0.

[0174] like Figure 2 As shown, three different cutoff strategies are typically used for data augmentation: Token Cutoff, Feature Cutoff, and SpanCutoff. This invention uses only Token Cutoff and Feature Cutoff. Token Cutoff randomly selects a Token and sets the entire row of the Token's Embedding to zero; or Feature Cutoff randomly selects a Feature from the Embedding and sets the entire column of the selected Feature dimension to zero.

[0175] (4) Dropout: This is a widely used regularization method to avoid overfitting. However, in relevant experiments, researchers have also demonstrated its effectiveness as an enhancement strategy for contrastive learning. In this setting, elements are randomly placed in the labeled embedding layer with specific probabilities and their values ​​are set to zero. Note that this strategy differs from "Cutoff" because each element is considered individually.

[0176] Feedforward artificial neural networks use non-linear "hidden" units between the input and output. By adjusting the weights of the incoming connections of these hidden units, it learns a feature detector that can predict the correct output given an input vector (1). If the relationship between the input and the correct output is complex, the network has enough hidden units to accurately model it, and there are often many different weight settings that can make the model almost perfect on the training set, especially if there is only limited labeled training data. This situation can lead to overfitting, affecting the model's actual performance and generalization ability.

[0177] Overfitting can be prevented by using "Dropout" to avoid complex co-adaptation of the training data. In each presentation of each training case, each hidden unit is randomly omitted with a probability of 0.5, thus preventing the hidden unit from depending on the presence of other hidden units. This method can also be used to generate inconsistent, similar samples, achieving data augmentation.

[0178] 2.4. Comparison of Loss Layers

[0179] This layer is defined as a contrastive prediction task. Given a set , including a pair of positive samples and The purpose of the contrast prediction task is to predict a given... ,exist Identified in .

[0180] From the same enhancement set ( and Two separate data augmentation operators are sampled and applied to each data sample to obtain two related sample pairs. These sample pairs are then used to augment a basic encoder network. and a projection neural network Perform training to maximize consistency using contrastive loss. After training, remove the projection head. and the encoder and represents Used for downstream tasks.

[0181] By random selection A batch of samples is used, and for the enhanced sample pairs in the mini-batch, a contrastive prediction task is defined to obtain... Given one data point and another positive sample pair, the remaining data points in a small batch are... Augmented samples are treated as negative samples.

[0182] pass To indicate through After regularization and The cosine similarity.

[0183] This leads to a positive example. The loss function is defined as shown in equation (12):

[0184] (12);

[0185] in 1 [ k ≠ a ] ∈ { 0 , 1 } It is if and only if An index function that takes a value of 1 at a given time. This represents the temperature parameter. The final loss function is calculated on all positive samples, simultaneously within a small batch of samples. and This function is called NT-Xent, which stands for Regularized Temperature-Scaled Cross-Entropy Loss Function.

[0186] 3. Detection Model

[0187] This invention designs a SED model for identifying and detecting sensitive euphemisms based on sentence representation. The SED model constructs word embeddings based on the ConSERT model and integrates a Bi-LSTM network with an attention mechanism, enabling it to effectively detect sensitive euphemisms in social networks. The SED model structure is as follows: Figure 3 As shown.

[0188] 3.1. Bi-LSTM

[0189] In the word embeddings extracted by BERT, each character in the text's sentence is mapped to a 768-dimensional vector. After average pooling and data augmentation, the entire sentence's 768-dimensional vector is obtained. The contrastive loss layer adjusts the sentence's vector representation using a contrastive loss function. Finally, the resulting sentence representation is input into a Bi-LSTM network. The Bi-LSTM network has a memory function, utilizing not only past information but also capturing subsequent information. The SED model uses Bi-LSTM for sentence feature extraction to avoid the gradient vanishing problem caused by RNN models in long sentences, and to extract contextual information and synthesize features, thereby improving the effectiveness of sensitive euphemism detection. After further feature extraction by the Bi-LSTM network, each sentence becomes a 128-dimensional vector.

[0190] Specifically, the model embeds the extracted sentences The input is fed into a Bi-LSTM network, which leverages its memory function to further learn the contextual features of the text, thereby obtaining a one-dimensional semantic feature vector. . The number of hidden nodes in the network. The calculation method is shown in equations (13)-(15).

[0191] (13);

[0192] (14);

[0193] (15);

[0194] in, It is the hidden vector generated by the previous state of the LSTM. It is the hidden vector generated by the next state of the LSTM.

[0195] 3.2 Attention Mechanism

[0196] After processing the sentence vectors through a Bi-LSTM layer, they are fed into an attention mechanism. Since different words contribute differently to the detection of sensitive euphemisms, this invention uses an attention mechanism to automatically discover words and features that play a key role in the classification of sensitive euphemisms and to capture the most important semantic information from each sentence. Sensitive euphemism detection models without an attention mechanism do not have problems when the input text is short, but if the input text is long, and all the semantics are represented by an intermediate semantic vector, then the information of the words themselves will be lost, resulting in the loss of a lot of detailed information. This is also an important reason for introducing an attention model in this invention. The attention mechanism will give higher weights to specific sensitive euphemisms, thereby improving the effect of sensitive euphemism detection, as shown in equations (16)-(18).

[0197] (16);

[0198] (17);

[0199] (18);

[0200] in, and Represents the weight matrix and bias terms. for The intermediate hidden layer vector. It is a randomly initialized vector that has already been optimized as model parameters during training. Weights Implemented by the Softmax function, it can be achieved by calculating the intermediate vector. and The similarity is used to determine the output vector. weight .also, This is the vector used to determine whether a sentence is a sensitive euphemism. `t` represents the top layer. represents the scaling factor, which is the hyperbolic tangent function.

[0201] 3.3 Output Layer

[0202] Finally, this invention reduces the dimension of the sentence vector trained by bidirectional LSTM and attention mechanism, puts it into Sigmoid to obtain the probability that the sentence vector is a sensitive euphemism, and thus obtains the final classification result, as shown in Equation (19).

[0203] (19);

[0204] in, This is the vector used to ultimately determine whether a sentence is a sensitive euphemism. This represents the probability that the sentence is a sensitive euphemism.

[0205] The optimization objective of the model is to minimize the cross-entropy loss function, as shown in equation (20).

[0206] L = − ∑ d ∈ D [ y d l o g p d + ( 1 − y d ) l o g ( 1 − p d ) ] (20);

[0207] in, Represents the sample dataset, Indicates a sample, Represents the true value of the sample. For the sample The probability of predicting a positive class. Minimize the cross-entropy loss for the objective.

[0208] 4. Experimental Verification

[0209] This invention designs three experiments to evaluate the detection performance of the SED model. All experiments were conducted in a server environment equipped with two Tesla-V100 32G GPUs. The dataset used in this experiment is the SES-Dataset constructed in this invention, and the relevant information of the dataset is shown in Table 1.

[0210] In the experiment, 70% of the SES dataset was used as the training set, 10% as the validation set, and 20% as the test set. Each experiment was run 10 times, and the average value was taken as the final result. The training, validation, and test sets used in the 10 experiments were randomly divided. This invention uses four metrics to evaluate the performance of the detection model, including accuracy, precision, recall, and F1 score. True Positive (TP) is the number of sentences predicted as sensitive euphemisms, False Negative (FN) is the number of sentences predicted as non-sensitive euphemisms, False Positive (FP) is the number of non-sensitive euphemisms predicted as sensitive euphemisms, and True Negative (TN) is the number of non-sensitive euphemisms predicted as non-sensitive euphemisms.

[0211] 4.1. Evaluate the effectiveness of the detection model

[0212] To evaluate the significant advantages of the proposed SED model in sensitive euphemism detection, comparative experiments were conducted on commonly used baseline models for sentence classification tasks, including traditional machine learning and deep learning, such as FastText, TextCNN, TextRCNN, TextRNN, TextRNN_Att, and the proposed SED model. The models were compared on four metrics: accuracy, precision, recall, and F1 score.

[0213] The SES-Dataset dataset was divided into validation, test, and training sets in a 1:2:7 ratio. The learning rate, training batch size, and number of epochs were fixed, and the random shuffle value of the dataset for each training iteration was also fixed. Due to the small size of the dataset and the high randomness of each training iteration, the average of multiple training results was used as the final result. Table 2 shows a comparison of the detection model performance. The model's effectiveness is as follows: Figure 4 As shown, compared to the baseline model, the SED model achieves a 5% improvement in accuracy compared to TextRCNN and approximately a 10% improvement compared to other baseline models; in terms of precision, the SED model improves by 1% to 10% compared to the baseline model; in terms of recall, the SED model reaches 91%, an 11% improvement compared to the second-ranked TextRCNN and a 12-21% improvement compared to other baseline models; the SED model's F1 score also exceeds that of the baseline model by 6% to 17%. The experiments demonstrate that the SED model achieves superior results in detection accuracy, precision, recall, and F1 score. Compared to general detection models, the SED model can more accurately detect and identify euphemisms.

[0214] Table 2 Comparison of performance of different detection models

[0215]

[0216] 4.2. The effect of evaluating sentence embedding

[0217] The semantic feature extraction module of the SED model uses the ConSERT model to generate sentence representations. To evaluate the performance of ConSERT sentence representations, this invention designs experiments to compare two common pre-trained word vector methods in the NLP field: context-related word embeddings (including BERT, Embeddings from Language Models (ELMo), etc.) and context-independent word embeddings (including Word2vec, fastText, etc.). Sentence representations are obtained by average pooling the word vectors. In the experiments, this invention uses five sentence vector embedding models—Sentence-BERT, Sentence-RoBERTa, Word2vec + average pooling, fastText + average pooling, and ConSERT—as the sentence embedding module of the SED model, while the structure of the remaining parts remains unchanged.

[0218] (1) Embedding of context-related sentences

[0219] Context-dependent word embeddings involve pre-trained models that can compute sentence embeddings based on the context of words. In this work, the present invention compares the following context-dependent word embedding models:

[0220] 1) Sentence-BERT: The BERT pre-trained model using the Siamese framework is then pooled at the output layer to generate sentence vectors. It supports multiple language embeddings and generates sentence vectors with a dimension of 768. The pre-trained model used in this experiment is distilbert-base-nli-stsb-mean-tokens.

[0221] 2) Sentence-RoBERTa: The RoBERTa pre-trained model using the Siamese framework is then pooled at the output layer to generate sentence vectors. It supports multiple language embeddings and generates sentence vectors with a dimension of 768. The pre-trained model used in this experiment is Roberta-base-nli-stsb-mean-tokens.

[0222] 3) ConSERT: This uses a BERT pre-trained model based on the simCLR framework, and improves sentence representation accuracy through data augmentation layers and contrastive loss layers. It is trained on multiple corpora and can be fine-tuned through unsupervised training on small-scale unlabeled corpora based on practical applications. Alternatively, it can be jointly trained using contrastive and supervised loss. The pre-trained model used in this experiment is unsup-consert-base-atec_ccks.

[0223] (2) Context-independent sentence embedding

[0224] In contrast to context-related sentence embedding, context-independent sentence embedding does not consider the contextual information of the words contained in the sentence. It directly generates the same word embedding vector for the same words in the vocabulary, and then performs pooling to obtain the sentence embedding vector. In this experiment, the present invention considers the following context-independent word embedding model:

[0225] 1) fastText: fastText uses word vectors trained by Facebook on the Wikipedia corpus, and the extracted character vectors have a dimension of 300.

[0226] 2) Word2vec: Word2vec uses Word2vec word vectors trained on the ChineseT CoNLL17 corpus by Oslo University, and the extracted character vectors have a dimension of 100.

[0227] The information for the four sentence embedding models is shown in Table 3.

[0228] Table 3 Information on four sentence embedding models

[0229]

[0230] In the actual implementation of the sentence embedding layer of this invention, to achieve better detection results for sensitive euphemisms, the ConSERT pre-trained model was further optimized. Since most models for sentence representation learning based on sentence similarity assessment tasks require pre-training, to ensure fairness and meet practical requirements, the other pre-trained sentence embedding models used in this experiment are all commonly used sentence embedding pre-training models. Experimental results are as follows... Figure 5 As shown in Table 4, overall, in the sensitive euphemism detection task, the context-related sentence embedding model outperforms the context-independent model. This is because the context-related sentence embedding model captures the appropriate meaning of words based on contextual information, while the context-independent model uses the same representation for words with different meanings. Furthermore, the SED model using the ConSERT sentence embedding model achieves better results than the Sentence-BERT and Sentence-RoBERTa sentence embedding models. This is because the ConSERT model uses various data augmentation patterns and a contrastive loss function to improve representation accuracy. Compared to other pre-trained sentence embedding models, ConSERT has a more significant advantage in terms of accuracy and discriminative power in sentence representation.

[0231] Table 4. Performance Comparison of Different Sentence Embedding Models

[0232]

[0233] Table 5 Comparison of Model Results

[0234]

[0235] 4.3. Evaluate the effectiveness of the proposed classification network

[0236] To evaluate the superior performance of the Bi-LSTM+attention mechanism classification network used in the proposed SED model in the sensitive euphemism detection task, this invention conducted comparative experiments with baseline models of commonly used sentence classification networks, including traditional machine learning and deep learning, all using ConSERT pre-trained models for sentence embedding representation. Five classification networks were compared: FNN, LSTM, LSTM+Att, Bi-LSTM, and BiLSRM+Att (SED). Comparisons were made on four metrics: accuracy, precision, recall, and F1 score. The comparison of detection performance among different classification networks is shown in the figure below. Figure 6 As shown.

[0237] Similarly, the SES-Dataset dataset was divided into validation, test, and training sets in a 1:2:7 ratio. The relevant training parameters were fixed, and the dataset was randomly shuffled for each training iteration. Due to the small size of the dataset and the high randomness of each training iteration, the same training parameters were used, and each model was trained ten times, with the average value taken as the final result.

[0238] The detailed information of the comparison model involved in this experiment is as follows:

[0239] (1) C+FNN (ConSERT+FNN): Uses a fully connected neural network (three linear layers + ReLU + Dropout).

[0240] (2) C+LSTM (ConSERT+LSTM): Using LSTM network.

[0241] (3) C+LSTM_Att (ConSERT+LSTM+Attention): Uses a unidirectional LSTM network and attention mechanism.

[0242] (4) C+Bi-LSTM (ConSERT+Bi-LSTM): Using Bi-LSTM network.

[0243] (5) SED (ConSERT+Bi-LSTM+Attention): This invention proposes a sensitive euphemism detection model using Bi-LSTM and attention mechanism.

[0244] In summary, the SED model proposed in this invention improves the final sensitive euphemism detection results by utilizing a contrastive learning framework, sentence embedding in the ConSERT model, employing a Bi-LSTM network, and introducing an attention mechanism. Therefore, the SED model of this invention achieves excellent results in detecting sensitive euphemisms in social networks.

Claims

1. A method for detecting sensitive euphemisms in social networks based on supervised learning, characterized in that, The overall process includes the following steps: Step 1: Data collection and annotation: Web crawlers are used to collect data from the social network platforms to be detected, and the collected data is filtered and selected, and then manually annotated to build a sensitive euphemism dataset; Step 2: Feature extraction: After initializing the embedding of the input sentence, an initial sentence vector is generated. Different augmented versions of the sentence vector are generated through different data augmentation methods. Then, BERT shared encoding is used to share the sentence vector generation parameters. Then, a contrastive learning layer is used for optimization and adjustment to make the spatial feature distribution of the sentence vector uniform. Finally, the augmented sentence vector is output. Step 3: Detection Model: Based on the ConSERT model and enhanced sentence vectors, and combined with Bi-LSTM and attention mechanism, a sensitive euphemism classification and detection model is constructed. The enhanced sentence vectors are input into the sensitive euphemism classification and detection model, and sensitive euphemisms on social network platforms are detected based on sentence feature vectors. Step 2 specifically includes: Step 2.1: Input the preprocessed data text through the input layer. ; Preprocessed sentences Include One word, The maximum sequence length set will exceed [the limit]. Discard any part of the character; add zeros if necessary. Step 2.2: Generate initial sentence vectors by processing the input layer's text data using the optimized BERT model; Step 2.2.1: Pre-train the BERT-Base-Chinese model using video comments and bullet screen data from the aforementioned social network platform, and fine-tune the model using the SES-Dataset dataset to obtain the optimized BERT model; Step 2.2.2: Transfer the preprocessed data text After tokenization, the tokenized text is obtained. The input is then fed into the optimized BERT model, where word embeddings are extracted and average pooling is applied to obtain high-quality semantic features in the form of sentence embeddings. ; as shown in the following formula: （2）； （3）； （4）； Where S is the sentence vector; Step 2.3: Generate sample pairs for contrastive learning using different data augmentation strategies through the data augmentation layer; Step 2.4: Complete the contrastive prediction task using the contrastive loss layer; Given a set , including a pair of positive samples and The purpose of the contrast prediction task is to predict a given... ,exist Identified in ; Two separate data augmentation operators are sampled from the same augmentation set and applied to each data sample to obtain two related sample pairs; using the obtained sample pairs, a basic encoder network is then applied. and a projection neural network Conduct training, and remove the projector head after training is complete. and the encoder and represents Used for downstream tasks; By random selection For batches of samples, and for small batches of augmented sample pairs, define a contrastive prediction task to obtain... Given one data point; then, given a positive sample pair, the other data points in a small batch... Augmented samples are treated as negative samples; This yields positive sample pairs. The loss function, namely the regularized temperature-scaled cross-entropy loss function, is defined as follows: （5）； in, It is if and only if An index function that takes a value of 1 at a given time. Represents temperature parameter; Indicates passage After regularization and cosine similarity, , and Samples ,sample and samples vector.

2. The method for detecting sensitive euphemisms in social networks based on supervised learning according to claim 1, characterized in that, Step 1 specifically includes: Step 1.1: Design a web crawler based on the Scrapy framework that supports multi-process operation for targeted collection of video comments and bullet comments on the social network platform to be detected; Step 1.2: In multiple channel sections of the social network platform, select a popular video with more than a predetermined number of video interactions as the seed video, perform breadth-first traversal, select and filter videos based on the video interaction number, and repeat the traversal to finally obtain the crawled dataset sample. Step 1.3: Define harmless everyday words that express sensitive content through malicious misinterpretation as sensitive euphemisms. Based on the content and context of the sentence, and referring to the existing sensitive euphemism database, manually label sensitive euphemism sentences that may express similar meanings to specific sensitive words.

3. The method for detecting sensitive euphemisms in social networks based on supervised learning according to claim 1, characterized in that, The aforementioned principles for manually annotating sensitive and euphemistic sentences specifically include: 1) If a sensitive euphemism can be reasonably interpreted according to its harmless meaning, but cannot be reasonably interpreted according to its sensitive meaning, then the sentence is considered to contain only harmless meaning, is marked as a harmless sentence, and the tag is set to 0; 2) If a sensitive euphemism cannot be reasonably interpreted according to its harmless meaning, but can be reasonably interpreted according to its sensitive meaning, then the sentence is considered to contain only the sensitive meaning, is marked as a sensitive euphemism, and the label is set to 1; 3) If the semantic information of a sentence only contains auxiliary words and sensitive euphemisms, and can be interpreted by both harmless and sensitive meanings, then the sentence is considered to contain both sensitive and harmless meanings, and is marked as a sensitive euphemism sentence with a label set to 1. 4) If a sensitive euphemism cannot be reasonably interpreted according to its harmless meaning, and also cannot be reasonably interpreted according to its sensitive meaning, then the sentence is considered semantically unclear, filtered out, and not tagged.

4. The method for detecting sensitive euphemisms in social networks based on supervised learning according to claim 1, characterized in that, Step 1 also includes evaluating the annotation effect of the manually annotated dataset, the specific steps of which are as follows: Two annotators independently browsed all the sentences to be labeled in the dataset and completed the data annotation independently. The results were then calculated... Coefficients are used to evaluate the consistency of the annotation results: （1）； in, , ; The set of sentences annotated by the first annotator. The set of sentences annotated by the second annotator; This is a set of sentences that the first annotator could not determine whether they were sensitive euphemisms. This is a set of sentences that the second annotator could not determine whether they are sensitive euphemisms. It is the collection of all sentences. It is the size of a set.

5. The method for detecting sensitive euphemisms in social networks based on supervised learning according to claim 2, characterized in that, The data augmentation strategy in step 2.3 includes adversarial attacks, specifically including: set up It is a parameter of a model. It is the input of the model, y is the... Related goals, It is the loss function used to train neural networks; the loss function is wrapped around... The current value is linearized as follows: （6）； In the formula, To improve the loss value, This is the learning rate or learning proportion parameter. For gradient calculation; The required gradients are calculated using backpropagation to train a single model. To identify labels , If it is the sigmoid function, then the training process with gradient descent is as follows: （7）； in, This is a symbolic representation of the gradient descent training process for the target data. It is the Softplus function; These are the weighting coefficients. This is the bias coefficient; The sign of the gradient is just... ,and , Let be the weight vector; then the adversarial version of logistic regression is to minimize the following equation: （8）； in, This is the inner product of the weight vectors when the prediction probability is 1.

6. The method for detecting sensitive euphemisms in social networks based on supervised learning according to claim 2, characterized in that, The data augmentation strategy in step 2.3 includes position shuffling, specifically including: The input sentence is shuffled by manipulating its position and sentence embedding. After encoding the shuffled input using BERT, the context embedding is considered. This corresponds to the insertion into the input sequence. Each sentence token, and the first one Token and the last one The token is passed to the sequence reconstructor to reconstruct the original sentence order. Except for the last one Token outside, The embeddings in the code are processed sequentially by the Transformer decoder to obtain the output. Simultaneously consider the entire And the embeddings processed in the previous steps Each processed output will be used in the prediction step. Sentences in As shown in the following formula: （9）； In the formula, For the sentence embedding of the i-th sentence; The probability distribution on the sentence representation is computed using a pointer network, where each probability represents the likelihood of each sentence appearing in the first step, as shown in the following formula: （10）； Calculate a cross-entropy loss to match sentences for each context, where the probability distribution of the embedded predicted order is related to the golden position; It is a sentence Appearing in location The predicted probability, Steps A label vector for the correct location in the middle; finally, the loss of all locations is averaged as As shown in the following formula: （11）； In the formula, For the average cross-entropy loss, The sentence position number, This is the label vector for the corresponding position in the sentence. For sentences Appearing in location The predicted probability; The model uses only sentence representations to find the original sequence, forcing the encoder to embed all necessary information into the context sentence embeddings instead of expanding them into all embeddings, as shown in the following equation: （12）； In the formula, For the modified cross-entropy loss, This represents the minimum cross-entropy loss.

7. The method for detecting sensitive euphemisms in social networks based on supervised learning according to claim 2, characterized in that, The data augmentation strategies in step 2.3 include Cutoff and Dropout, specifically including: Cutoff can be implemented in two ways: using Token Cutoff, randomly selecting a Token and setting the entire row of the corresponding Token's Embedding to zero, or using Feature Cutoff, randomly selecting a Feature from the Embedding and setting the entire column of the selected Feature dimension to zero. Dropout specifically involves randomly placing elements in the tag embedding layer with a specific probability, setting their values ​​to zero, and considering each element individually.

8. The method for detecting sensitive euphemisms in social networks based on supervised learning according to claim 2, characterized in that, Step 3 specifically includes: Step 3.1: Extract the sentence vectors The input is fed into a Bi-LSTM network, which leverages its memory function to further learn contextual features, thereby obtaining a one-dimensional semantic feature vector. ; The number of hidden nodes in the network. The calculation method is shown in the following formula: （13）； （14）； （15）； in, It is the hidden vector generated by the previous state of the LSTM. It is the hidden vector generated by the next state of the LSTM. Word vectors; Step 3.2: The sentence vectors processed by the Bi-LSTM layer are fed into the attention mechanism. Specific sensitive euphemisms are assigned higher weights to improve the detection performance of sensitive euphemisms, as shown in the following formula: （16）； （17）； （18）； in, and Represents the weight matrix and bias terms. For the output vector The intermediate hidden layer vector; It is a randomly initialized vector that has already been optimized as model parameters during training; weights Implemented by the Softmax function, it calculates the intermediate vector. and The similarity is used to determine the output vector. weight ;also, The vector used to determine whether a sentence is a sensitive euphemism is used; t represents the top layer. This represents the scaling factor, which is the hyperbolic tangent function; Step 3.2: Reduce the dimensionality of the sentence vectors trained by the Bi-LSTM network and attention mechanism, and feed them into the Sigmoid function to obtain the probability that the sentence vector is a sensitive euphemism. This yields the final classification result, as shown in the following formula: （19）； in, This represents the probability that the sentence is a sensitive euphemism.

9. The method for detecting sensitive euphemisms in social networks based on supervised learning according to claim 1, characterized in that, The optimization objective of the sensitive euphemism classification and detection model is to minimize the cross-entropy loss function, as shown in the following equation: （20） in, Represents the sample dataset, Indicates a sample, Represents the true value of the sample. For the sample The probability of predicting a positive class. Minimize the cross-entropy loss for the objective.

Images