Method for detecting hidden hatred language based on soft prompt tuning

Abstract

The invention discloses a hidden hatred language detection method based on soft prompt tuning, which comprises the following steps: 1) training in a continuous optimal prompt space by using a soft prompt template; and 2) aiming at differences of Chinese and English in semantic structures and grammatical rules, constructing an expansion word mapper by using different methods. And 3) combining the soft prompt template and the expansion word mapper mentioned in the step 1) and the step 2), and selecting a small amount of samples from the training set for training to construct a prompt tuning model so as to realize hidden hatred language detection. According to the method, three different strategies are adopted to expand a label word space so as to explore a real intention behind an original text, integration of the strategies is used in final prompt tuning, and the method can further stimulate rich knowledge distributed in a pre-trained language model and can be better applied to detection of the recessive hatred language.

Description

Technical Field

[0001] The present invention relates to the research field of implicit hate language and short text classification, and in particular to an implicit hate language detection method based on soft prompt tuning. Background Art

[0002] With the rapid development of the Internet, users can express their opinions freely without using real information. However, the accompanying hate language is widely spread on social media, which may further develop into "cyber violence" and aggravate harmful consequences. Recently, some studies have used dictionary-based or deep learning methods to detect hate language, but when faced with hate language without obvious insults, it is usually impossible to detect implicit hatred or insults. Research on explicit and implicit hate language detection has evolved from early feature engineering methods to neural networks, and more recently to pre-trained language models such as BERT and HateBERT, which have also shown advantages in the task of hate language detection.

[0003] Faced with the huge challenges brought by implicit hate language, existing detection methods are unable to achieve satisfactory results for the nuances and diversity of language. The problems that hinder the further development of existing methods are summarized from two aspects: (1) Nuances and high ambiguity of language: Since implicit abuse introduces low keyword co-occurrence and concept confusion, how to accurately discover the true meaning behind the text remains a big challenge. (2) Language diversity and feature sparsity: Implicit hate language is mostly comments, free expression, and basically only a few dozen words of short text, such as Twitter or Weibo. It inevitably leads to language diversity and feature sparsity, which reduces the performance of the model. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the prior art and provide a hidden hate language detection method based on soft prompt tuning, which uses automatically generated soft prompt templates and combines three different strategies to expand the label word space to explore the true intention behind the original text; regards network comments that basically consist of only a dozen words as short texts, thereby converting the hidden hate language detection task into a short text classification task, thereby improving the performance of hidden hate language detection.

[0005] The object of the present invention is achieved by: a method for detecting implicit hate language based on soft prompt tuning, comprising the following steps:

[0006] 1) Use soft prompt templates and train in a continuous optimal prompt space;

[0007] 2) In view of the differences between Chinese and English in semantic structure and grammatical rules, different methods are used to construct expansion word mappers; the construction method for English is to extract adjectives from the training set and remove the disturbed expansion words through three additional strategies; when processing Chinese, the expansion words are obtained through synonyms and spectral clustering algorithms;

[0008] 3) Combining the soft prompt template and the extended word mapper mentioned in step 1) and step 2), and selecting a small number of samples from the training set for training to construct a prompt tuning model to achieve implicit hate language detection.

[0009] As a further limitation of the present invention, the step 1) specifically comprises:

[0010] Step 1.1) Design a soft template based on the implicit hate language detection method of soft prompt tuning and train it in a continuous optimal prompt space, which is expressed as:

[0011] T={[u i ],...,x,...,[u n ],[mask]} (1)

[0012] where u i represents the i-th learnable marker;

[0013] The constructed hint template T is then fed into the encoder of the pre-trained language model to generate the hidden vector h i ,...,h x ,...,h n ,h mask ; At this time, the soft prompt template T is expressed as:

[0014] T={[h i ],...,h x ,...,[h n ],[h mask ]} (2)

[0015] Step 1.2) Introduce the BiLSTM model as the neural network for the hidden vector h i Modeling is performed, which is expressed as:

[0016]

[0017] Among them, h i represents the hidden vector of the i-th position in the sequence, h i-1 and h i+1 Represents the hidden vectors of adjacent positions, h0 and h nRepresents the initial hidden vector of the forward LSTM and the backward LSTM; the forward LSTM and the backward LSTM represent bidirectional LSTM, that is, the bidirectional long short-term memory network BiLSTM;

[0018] Finally, the model uses the loss function L to measure the gap between the output result of the model M when given the input x and mask and the target value, and selects the optimal variable value h. The loss function is as follows:

[0019]

[0020] Among them, M(x,mask) refers to the output result of model M when given input x and mask mask, and the loss function L is used to measure the gap between the predicted output of model M and the target value.

[0021] As a further limitation of the present invention, the step 2) specifically includes

[0022] Step 2.1) In the prompt tuning, the mapping from the expansion word to the category effectively alleviates the difference between the text and label space, and becomes the automatic selection of label words or the expansion word mapping, which is expressed as:

[0023]

[0024] Herev i represents a word in the extended word mapper; y represents the corresponding label, where y = 0 represents non-hate language, which is associated with the "friendly" label word, and on the contrary, y = 1 represents hate language, which corresponds to the "malicious" label;

[0025] Step 2.2) When constructing the English extension words, words are extracted independently from the data of the non-hate language and hate language parts of the training set; in the non-hate language part of the training set, three different part-of-speech tagging tools including NLTK, spaCy, and TextBlob are used to identify and retrieve adjectives and extract them, which are expressed as:

[0026] {a1,...,a i ,...,a n}=ExtractA{NL pos ,SC pos ,TB pos} (6)

[0027] Among them NL pos ,SC pos ,TB pos They represent the results of three POS tagging tools;

[0028] Then, the adjectives extracted by the three tools are used to construct three sets; the intersection of the three sets is taken using an intersection-based method to refine the word set. This process can be concisely expressed as:

[0029] S final =S NL ∩S SC ∩S TB (7)

[0030] Where “∩” represents the intersection operation, S final represents the result set of expanded words obtained by intersection; the same method is also used for the hate language part of the training set to achieve comprehensive coverage and refinement;

[0031] Step 2.3) After obtaining S final Finally, due to the large number of online review texts and the introduction of noise and irrelevant terms, in order to capture the different aspects of the extended words and explore the true intention behind the original text, three strategies are further adopted to optimize the extracted adjectives. The three strategies include probability prediction, FastText similarity and context information;

[0032] Step 2.4) According to the three strategies in step 2.3), the expanded words are denoised, and the union of the expanded words optimized by the three strategies is taken as the final expanded words for English implicit hate language detection;

[0033] Step 2.5) Due to the differences between Chinese and English in semantic structure and grammatical rules, the Chinese language presents extremely complex semantics and grammatical rules that interact with each other. A series of synonyms are selected on the Chinese synonym website for the non-hate language tag word "friendly" and the hate language tag word "malicious", and spectral clustering is performed on this series of synonyms to select the N words closest to the tag word to form the Chinese extended word mapper.

[0034] As a further limitation of the present invention, the step 2.3) specifically includes:

[0035] Step 2.3.1) Probabilistic prediction is a strategy to build an extended word mapper using the rich knowledge contained in the pre-trained language model, that is, using the BERT model to capture the dependencies between words, allowing the representation of each word to simultaneously contain information from its context; fine-tune the task model for detecting hate language to obtain the probability distribution associated with the “[MASK]” token in its template, based on these probabilities, sort all words in descending order, and select the top N words;

[0036] Step 2.3.2) The extended word mapper refers to the mapping from extended words to corresponding categories. First, the FastText word embedding model is introduced to obtain the vector representation of the word; then, the cosine similarity between the category label word and the original extracted extended word is calculated using the similarity metric; finally, the top N words with the highest similarity are selected, which is the same as the probability prediction; the similarity calculation is:

[0037]

[0038] Here ||v|| represents the norm of vector v;

[0039] Step 2.3.3) Consider the context of the expanded word, especially the words before and after the [MASK] tag, and use the BERT model to obtain contextual information; replace the [MASK] tag with the constructed expanded word and mask one of the surrounding words; then input the new text back into BERT to calculate the cross entropy loss; finally, sort the expanded words according to the corresponding losses, keep the first N words, and discard the remaining words.

[0040] As a further limitation of the present invention, the step 3) specifically comprises:

[0041] Step 3.1) Input text sequence x = [x1, x2, ..., x n ] is converted into the embedding required by the pre-trained language model. The embedding layer converts each x i Mapped into a high-dimensional embedding vector e i , and get the embedding representation matrix E:

[0042] E=[e1,e2,...,e n ]=Embed(x) (9)

[0043] Where Embed(x) represents the embedding function; then the embedding representation matrix E is processed by multiple layers of Transformer to obtain the encoded representation matrix:

[0044] H = Transformer (E) (10)

[0045] Transformer (E) represents the output after multi-layer processing of the Transformer structure, and the generated encoding sequence is represented by H = [h1,h2,...,h n ]; This process is responsible for understanding and encoding the semantic and contextual information of the input sequence. By adopting a mask model, the implicit hate language detection task is formalized as a cloze task;

[0046] Step 3.2) The final extended word mapper is obtained through various strategies and combined with the constructed automatic soft template, and the weighted average of the label word scores is used as the prediction score; that is, it is obtained by the weighted average of the label word scores of non-hate and hate language, and the weight coefficient is determined by the calculated weight of each word:

[0047]

[0048] Here V y represents the set of label words corresponding to label y, and |V y | indicates V y The function p([MASK]=vH) is used to calculate the probability of the label word v given the encoding representation matrix H.

[0049] The present invention adopts the above technical solution, and compared with the prior art, the beneficial effects are as follows: (1) The present invention can simultaneously consider the automatic generation and detection performance of templates; when expanding the label word space, both the comments and the real intentions behind the comments are considered, so that more ideal performance can be obtained; (2) The present invention is a simple, effective and scalable method for detecting implicit hate language. The method of the present invention omits many steps used in the existing hate language detection methods, such as co-occurrence word search and entire model tuning, and is simpler to implement; new state-of-the-art results are obtained on three data sets with high effectiveness; the strategy of expanding word integration can be expanded by multiple features and strategies, and the combination of these expanding words from different strategies can significantly improve the performance of implicit hate language detection, and has high scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The overall framework diagram of the present invention. DETAILED DESCRIPTION

[0051] like Figure 1 The method for detecting implicit hate language based on soft prompt tuning is shown, comprising the following steps:

[0052] 1) Use soft prompt templates and train in a continuous optimal prompt space;

[0053] Step 1.1) Compared with the manually designed manual template for prompt tuning method, the implicit hate language detection method based on soft prompt tuning proposed in this paper designs a soft template and trains it in a continuous optimal prompt space, which can be expressed as:

[0054] T={[u i ],...,x,...,[u n ],[mask]} (1)

[0055] where u irepresents the i-th learnable marker;

[0056] The constructed hint template T is then fed into the encoder of the pre-trained language model to generate the hidden vector h i ,...,h x ,...,h n ,h mask At this time, the soft prompt template T can be expressed as:

[0057] T={[h i ],...,h x ,...,[h n ],[h mask ]} (2)

[0058] Step 1.2) Introduce the BiLSTM model as the neural network for the hidden vector h i Modeling is performed, which is expressed as:

[0059]

[0060] Among them, h i represents the hidden vector of the i-th position in the sequence, h i-1 and h i+1 Represents the hidden vectors of adjacent positions, h0 and h n Represents the initial hidden vector of the forward LSTM and the backward LSTM; the forward LSTM and the backward LSTM represent bidirectional LSTM, that is, the bidirectional long short-term memory network BiLSTM;

[0061] Finally, the model uses the loss function L to measure the gap between the output result of the model M when given the input x and mask and the target value, and selects the optimal variable value h. The loss function is as follows:

[0062]

[0063] Among them, M(x,mask) refers to the output result of model M when given input x and mask mask, and the loss function L is used to measure the gap between the predicted output of model M and the target value.

[0064] 2) In view of the differences between Chinese and English in semantic structure and grammatical rules, different methods are used to construct expansion word mappers; the construction method for English is to extract adjectives from the training set and remove the disturbed expansion words through three additional strategies; when processing Chinese, the expansion words are obtained through synonyms and spectral clustering algorithms;

[0065] Step 2.1) In prompt tuning, the mapping from extended words (such as malicious, offensive, low-level, etc.) to categories (such as hate language) can effectively alleviate the difference between text and label space, becoming the automatic selection of label words or extended word mapping, which can be expressed as:

[0066]

[0067] Herev i represents a word in the extended word mapper; y represents the corresponding label, where y = 0 represents non-hate language, which is associated with the "friendly" label word, and on the contrary, y = 1 represents hate language, which corresponds to the "malicious" label;

[0068] In cue tuning, the process of mapping expanded words to corresponding categories has been shown to be effective in improving the performance of downstream tasks;

[0069] Step 2.2) When constructing the English extension words, words are extracted independently from the data of the non-hate language and hate language parts of the training set; in order to explore the true intention behind the original text, the detection of implicit hate language emphasizes the ability of adjectives to convey the nuances of language in terms of emotion; in the non-hate language part of the training set, three different part-of-speech tagging tools, including NLTK, spaCy, and TextBlob, are used to identify and retrieve adjectives and extract them, which can be expressed as:

[0070] {a1,...,a i ,...,a n}=ExtractA{NL pos ,SC pos ,TB pos} (6)

[0071] Among them NL pos ,SC pos ,TB pos They represent the results of three POS tagging tools;

[0072] Then, three sets are constructed from the adjectives extracted by the three tools. In order to improve the ability to capture online comment features for implicit hate language detection, and because the results obtained by various part-of-speech tools have inherent biases, an intersection-based method is used here to take the intersection of the three sets. The purpose is to refine the set of words, thereby minimizing the potential errors that may occur when using a single tool. This process can be concisely expressed as:

[0073] S final =S NL ∩S SC ∩S TB (7)

[0074] Where “∩” represents the intersection operation, S final represents the result set of expanded words obtained by intersection. The same method is also used for the hate language part of the training set to achieve comprehensive coverage and refinement;

[0075] Step 2.3) After obtaining S final Finally, due to the large number of online comment texts and the possible introduction of noise and irrelevant terms, in order to capture the different aspects of the extended words and explore the true intention behind the original text, three strategies are further adopted to optimize the extracted adjectives. The three strategies include probability prediction, FastText similarity and context information. The specific descriptions are as follows:

[0076] Step 2.3.1) Probability prediction is a strategy to build an extended word mapper using the rich knowledge contained in the pre-trained language model, that is, using the BERT model to capture the dependencies between words, allowing the representation of each word to simultaneously contain information from its context; fine-tune the task model for detecting hate language to obtain the probability distribution associated with the "[MASK]" tag in its template. Based on these probabilities, sort all words in descending order and select the top N words;

[0077] Step 2.3.2) Since the expansion word mapper refers to the mapping from expansion words to corresponding categories, the similarity between the expansion words and the category name y can be used to construct the expansion words; first, the FastText word embedding model is introduced to obtain the vector representation of the word; then, the cosine similarity between the category label word and the original extracted expansion word is calculated using the similarity metric; finally, the top N words with the highest similarity are selected, which is the same as the probability prediction. The similarity calculation is:

[0078]

[0079] Here ||v|| represents the norm of vector v;

[0080] Step 2.3.3) In order to consider the context of the expanded word, especially the words before and after the [MASK] tag, the BERT model is used to obtain contextual information; the [MASK] tag is replaced with the constructed expanded word, and one of the surrounding words is masked; then the new text is input back into BERT to calculate the cross entropy loss; finally, the expanded words are sorted according to the corresponding losses, the first N words are retained, and the remaining words are discarded;

[0081] Step 2.4) According to the three strategies in step 2.3), the expanded words are denoised, and the union of the expanded words optimized by the three strategies is taken as the final expanded words for English implicit hate language detection;

[0082] Step 2.5) Due to the differences between Chinese and English in semantic structure and grammatical rules, the Chinese language presents extremely complex semantic and grammatical rules that interact with each other, especially in areas such as word order and semantic ambiguity; therefore, targeted research was conducted on Chinese, and a series of synonyms were selected on the Chinese synonym website for the non-hate language label word "friendly" and the hate language label word "malicious", and spectral clustering was performed on this series of synonyms to select the N words closest to the label word to form an extended word mapper for Chinese.

[0083] 3) Combining the soft prompt template and the extended word mapper mentioned in step 1) and step 2), and selecting a small number of samples from the training set for training to construct a prompt tuning model to achieve implicit hate language detection;

[0084] Step 3.1) Input text sequence x = [x1, x2, ..., x n ] is converted into the embedding required by the pre-trained language model. The embedding layer converts each x i Mapped into a high-dimensional embedding vector e i , and get the embedding representation matrix E:

[0085] E=[e1,e2,...,e n ]=Embed(x) (9)

[0086] Where Embed(x) represents the embedding function. Then the embedding representation matrix E is processed by multiple layers of Transformer to obtain the encoded representation matrix:

[0087] H = Transformer (E) (10)

[0088] Transformer (E) represents the output after multi-layer processing of the Transformer structure, and the generated encoding sequence is represented by H = [h1,h2,...,h n ]; This process is responsible for understanding and encoding the semantic and contextual information of the input sequence. By adopting a mask model, the implicit hate language detection task is formalized as a cloze task;

[0089] Step 3.2) The final extended word mapper is obtained through various strategies and combined with the constructed automatic soft template, and the weighted average of the label word scores is used as the prediction score; it is obtained by the weighted average of the label word scores of non-hate and hate language, and the weight coefficient is determined by the calculated weight of each word:

[0090]

[0091] Here V y represents the set of label words corresponding to label y, and |Vy | indicates V y The function p([MASK]=vH) calculates the probability of the label word v given the encoding representation matrix H.

[0092] In order to verify the performance of the present invention in monitoring implicit hate language, experiments were conducted on two well-known public datasets and one self-constructed dataset. The two public datasets are "Latent Hatred" (LHd) and "Chinese Offensive Language Detection" (COLD). LHd is an English dataset obtained from online hate groups and their followers on Twitter. These collected comments are divided into "explicit hate", "implicit hate" and "non-hate". In this experiment, the data with the "explicit hate" label is deleted; COLD is a Chinese dataset that annotates hate language and non-hate language. The content in this dataset is obtained from platforms such as Zhihu and Weibo, and is annotated by combining manual annotation and machine learning training. In addition, this experiment selects comments from hotly discussed posts on Weibo to construct a Chinese dataset "Chinese Implicit Hate Speech" (CIHS), and uses manual annotation to better distinguish implicit hate language. Comments that show obvious hatred and implicit hatred are classified as "hateful", while well-intentioned or objective comments are classified as "friendly". The above three datasets are preprocessed and statistically analyzed to obtain Table 1.

[0093] Table 1 Dataset statistics

[0094] Statistic COLD CIHS LHd #total headline 31049 6699 11600 #train sets 25726 4690 7000 #test sets 5323 2009 4600 Arg.#words per sentence 46 21 88

[0095] In order to ensure that the detection effect can be truly measured, four representative evaluation indicators, Accuracy, Precision, Recall and F1, are selected. The specific definitions are as follows:

[0096]

[0097] Where tp is the number of malicious comments correctly predicted by the algorithm, fp is the number of comments predicted to be malicious but are in fact friendly, fn is the number of comments predicted to be friendly but are in fact malicious, tn is the number of friendly comments that have been correctly predicted, N is the total number of predicted web comments, and the F1 score is the harmonic mean of precision and recall.

[0098] In order to demonstrate the performance of the test results, other traditional baseline methods are used on three data sets to compare with the implicit hate language detection method based on soft prompt tuning (HIS_SP) of the present invention. These baseline methods include 1) feature engineering-based methods: support vector machine (SVM), random forest (RandomF); 2) deep neural network methods: bidirectional long short-term memory (BiLSTM), convolutional neural network (CNN), gated recurrent unit (GRU); 3) based on pre-trained language model: HateBERT, hate language detection method based on sentiment features (SKS); 4) prompt learning methods: prompt learning (PL), soft template prompt fine-tuning (Soft), prompt learning based on external knowledge (KPT), implicit hate language detection based on prompt learning (IHS_PL).

[0099] It is worth noting that in the experiment, 20 positive and negative samples were randomly selected in the training set to model the few-shot scenario in the hint learning method, while the number of positive and negative sample training based on feature engineering, deep neural network and pre-trained language model was 3200, 1600 and 1600 on the datasets COLD, CIHS and LHd respectively. The results of the dataset test are shown in Table 2, which shows that the method of the present invention outperforms other methods in all four indicators on the three datasets.

[0100] Table 2 Experimental results

[0101]

[0102] The present invention proposes a method for detecting implicit hate language based on soft prompt tuning. The method fully considers the characteristics of implicit hate language, such as linguistic subtlety, high fuzziness, language diversity and feature sparsity. While expanding the label word space, it can explore the true intention behind the original text. In view of the differences in Chinese and English features, different strategies are used to expand the label words of each category. Considering that manually designed templates are time-consuming and laborious and may not necessarily design the most suitable template for the corresponding detection task, the present invention uses automatically generated soft templates to detect implicit hate language. Experiments have proved the effectiveness of this method.

[0103] The present invention is not limited to the above-mentioned embodiments. On the basis of the technical solution disclosed in the present invention, technicians in this field can make some substitutions and deformations to some technical features therein according to the disclosed technical content without creative labor, and these substitutions and deformations are all within the protection scope of the present invention.

Claims

1. A hidden hate language detection method based on soft prompt tuning, It is characterized in that The following steps are involved: 1) Use soft prompt templates and train in a continuous optimal prompt space; 2) In view of the differences between Chinese and English in semantic structure and grammatical rules, different methods are used to construct expansion word mappers; the construction method for English is to extract adjectives from the training set and remove the disturbed expansion words through three additional strategies; when processing Chinese, the expansion words are obtained through synonyms and spectral clustering algorithms; 3) Combining the soft prompt template and the extended word mapper mentioned in step 1) and step 2), and selecting a small number of samples from the training set for training to construct a prompt tuning model to achieve implicit hate language detection.

2. According to claim 1, a hidden hate language detection method based on soft prompt tuning, It is characterized in that The step 1) specifically includes: Step 1.1) Design a soft template based on the implicit hate language detection method of soft prompt tuning and train it in a continuous optimal prompt space, which is expressed as: T={[u i ],...,x,...,[u n ],[mask]} (1) where u i represents the i-th learnable marker; The constructed hint template T is then fed into the encoder of the pre-trained language model to generate the hidden vector h i ,...,h x ,...,h n ,h mask ; At this time, the soft prompt template T is expressed as: T={[h i ],...,h x ,...,[h n ],[h mask ]} (2) Step 1.2) Introduce the BiLSTM model as the neural network for the hidden vector h i Modeling is performed, which is expressed as: Among them, h i represents the hidden vector of the i-th position in the sequence, h i-1 and h i+1 represents the hidden vector of the adjacent position, h 0 and h n Represents the initial hidden vector of the forward LSTM and the backward LSTM; the forward LSTM and the backward LSTM represent bidirectional LSTM, that is, the bidirectional long short-term memory network BiLSTM; Finally, the model uses the loss function L to measure the gap between the output result of the model M when given the input x and mask and the target value, and selects the optimal variable value h. The loss function is as follows: Among them, M(x,mask) refers to the output result of model M when given input x and mask mask, and the loss function L is used to measure the gap between the predicted output of model M and the target value.

3. According to claim 1, a hidden hate language detection method based on soft prompt tuning, It is characterized in that The step 2) specifically includes Step 2.1) In the prompt tuning, the mapping from the expansion word to the category effectively alleviates the difference between the text and label space, and becomes the automatic selection of label words or the expansion word mapping, which is expressed as: Herev i represents a word in the extended word mapper; y represents the corresponding label, where y = 0 represents non-hate language, which is associated with the "friendly" label word, and on the contrary, y = 1 represents hate language, which corresponds to the "malicious" label; Step 2.2) When constructing the English extension words, words are extracted independently from the data of the non-hate language and hate language parts of the training set; in the non-hate language part of the training set, three different part-of-speech tagging tools including NLTK, spaCy, and TextBlob are used to identify and retrieve adjectives and extract them, which are expressed as: {a 1 ,...,a i ,...,a n }=ExtractA{NL pos ,SC pos ,TB pos } (6) NL pos ,SC pos ,TB pos They represent the results of three POS tagging tools; Then, the adjectives extracted by the three tools are used to construct three sets; the intersection of the three sets is taken using an intersection-based method to refine the word set. This process can be concisely expressed as: S final =S NL ∩S SC ∩S TB (7) Where "∩" represents the intersection operation, S final represents the result set of expanded words obtained by intersection; the same method is also used for the hate language part of the training set to achieve comprehensive coverage and refinement; Step 2.3) After obtaining S final Finally, due to the large number of online review texts and the introduction of noise and irrelevant terms, in order to capture the different aspects of the extended words and explore the true intention behind the original text, three strategies are further adopted to optimize the extracted adjectives. The three strategies include probability prediction, FastText similarity and context information; Step 2.4) According to the three strategies in step 2.3), the expanded words are denoised, and the union of the expanded words optimized by the three strategies is taken as the final expanded words for English implicit hate language detection; Step 2.5) Due to the differences between Chinese and English in semantic structure and grammatical rules, the Chinese language presents extremely complex semantics and grammatical rules that interact with each other. A series of synonyms are selected on the Chinese synonym website for the non-hate language tag word "friendly" and the hate language tag word "malicious", and spectral clustering is performed on this series of synonyms to select the N words closest to the tag word to form the Chinese extended word mapper.

4. According to claim 3, a hidden hate language detection method based on soft prompt tuning, It is characterized in that The step 2.3) specifically includes: Step 2.3.1) Probabilistic prediction is a strategy to build an extended word mapper using the rich knowledge contained in the pre-trained language model, that is, using the BERT model to capture the dependencies between words, allowing the representation of each word to simultaneously contain information from its context; fine-tune the task model for detecting hate language to obtain the probability distribution associated with the "[MASK]" token in its template, based on these probabilities, sort all words in descending order, and select the top N words; Step 2.3.2) The extended word mapper refers to the mapping from extended words to corresponding categories. First, the FastText word embedding model is introduced to obtain the vector representation of the word; then, the cosine similarity between the category label word and the original extracted extended word is calculated using the similarity metric; finally, the top N words with the highest similarity are selected, which is the same as the probability prediction; the similarity calculation is: Here ||v|| represents the norm of vector v; Step 2.3.3) Consider the context of the expanded word, especially the words before and after the [MASK] tag, and use the BERT model to obtain contextual information; replace the [MASK] tag with the constructed expanded word and mask one of the surrounding words; then input the new text back into BERT to calculate the cross entropy loss; finally, sort the expanded words in ascending order according to the corresponding loss, retain the first N words, and discard the remaining words.

5. According to claim 1, a hidden hate language detection method based on soft prompt tuning, It is characterized in that The step 3) specifically includes: Step 3.1) Input text sequence x = [x 1 ,x 2 ,...,x n ] is converted into the embedding required by the pre-trained language model. The embedding layer converts each x i Mapped into a high-dimensional embedding vector e i , and get the embedding representation matrix E: E=[e 1 ,e 2 ,...,e n ]=Embed(x) (9) Where Embed(x) represents the embedding function; then the embedding representation matrix E is processed by multiple layers of Transformer to obtain the encoded representation matrix: H = Transformer (E) (10) Transformer (E) represents the output after multi-layer processing of the Transformer structure, and the generated encoding sequence is represented by H = [h 1 ,h 2 ,...,h n ]; This process is responsible for understanding and encoding the semantic and contextual information of the input sequence. By adopting a mask model, the implicit hate language detection task is formalized as a cloze task; Step 3.2) The final extended word mapper is obtained through various strategies and combined with the constructed automatic soft template, and the weighted average of the label word scores is used as the prediction score; that is, it is obtained by the weighted average of the label word scores of non-hate and hate language, and the weight coefficient is determined by the calculated weight of each word: Here V y represents the set of label words corresponding to label y, and |V y | indicates V y The function p([MASK]=v|H) calculates the probability of the label word v given the encoding representation matrix H.

Images