A hate speech detection method, device and computer equipment based on contrast learning

Abstract

This invention provides a method, apparatus, and computer device for hate speech detection based on contrastive learning. The method includes: constructing a hate speech dataset based on preset sensitive keywords and a dialogue generation model; building a hate speech detection model and training it using the hate speech dataset; wherein the hate speech detection model sequentially includes a word embedding layer, a semantic extraction layer, a contrastive mapping layer, a contrastive loss layer, and an output layer; inputting the speech to be detected into the trained hate speech detection model and outputting the detection result of the speech to be detected. The contrastive learning-based hate speech detection method provided by this solution improves the accuracy of hate speech detection.

Description

Technical Field

[0001] This invention relates to the field of data classification technology, and more particularly to the field of online speech governance technology, and especially to a method, apparatus and computer device for detecting hate speech based on contrastive learning. Background Technology

[0002] With the continuous growth of users on social media platforms, hate speech is not only surging in quantity but also spreading at a much faster pace, posing a significant challenge to social stability and individual mental health. Hate speech refers to language that contains insults, hatred, prejudice, discrimination, illegal and criminal content, sensitive topics, or physical harm. Hate speech can be expressed in various ways, ranging from overt insults and threats to subtle attacks. However, overly strict hate speech moderation mechanisms can significantly reduce the responsiveness of large-scale data models and the scope of user interaction with them.

[0003] To prevent the negative impact of hate speech on individuals and society, hate speech detection has become a research focus. Currently, there are many methods for hate speech detection, including rule-based keyword filtering, machine learning-based methods (such as Support Vector Machines (SVM), decision trees, and random forests), and deep learning-based methods (such as hate speech detection based on pre-trained models). However, these methods cannot fully extract semantic information from short texts, and the difference between positive and negative hate speech samples is very small, resulting in low accuracy in hate speech detection. Summary of the Invention

[0004] To address the aforementioned issues, embodiments of the present invention provide a hate speech detection method, apparatus, and computer device based on contrastive learning, which improves the accuracy of hate speech detection.

[0005] In a first aspect, embodiments of the present invention provide a hate speech detection method based on contrastive learning, comprising:

[0006] A dataset of hate speech dialogues was constructed based on preset sensitive keywords and a dialogue generation model.

[0007] A hate speech detection model is constructed and trained using the hate speech dataset; wherein, the hate speech detection model comprises, in sequence, a word embedding layer, a semantic extraction layer, a contrast mapping layer, a contrast loss layer, and an output layer;

[0008] The hate speech detection model inputs the speech to be detected into the trained model and outputs the detection result of the speech to be detected.

[0009] Optionally, the step of constructing a hate speech dataset based on preset sensitive keywords and a dialogue generation model includes:

[0010] Obtain different types of hate speech as positive samples;

[0011] Input preset sensitive keywords into the dialogue generation model to generate negative samples;

[0012] Based on the positive and negative samples, the hate speech dataset is constructed.

[0013] Optionally, the construction of the hate speech detection model includes:

[0014] The Roberta-WWM model is used as the word embedding layer; the MLP layer is used as the contrast mapping layer; and Bi-LSTM or Bi-GRU is used as the semantic extraction layer.

[0015] Optionally, the construction of the hate speech detection model includes:

[0016] The input text is obtained by concatenating the label information of each category before the text to be input; wherein, the categories include abnormal and normal.

[0017] The input text is input into the word embedding layer, and the output is a context vector, a sequence of label vectors for each category, and a word embedding vector matrix;

[0018] The label vector sequence of each category and the word embedding vector matrix are input into the semantic extraction layer, and several semantic feature vectors are output; the semantic feature vectors include label vectors output from the label vectors of each category;

[0019] The context vector, the label vector of the abnormal category, and the label vector of the normal category are input into the contrast mapping layer, and the first feature vector, the second feature vector, and the third feature vector are output respectively.

[0020] The first feature vector is compared with the second feature vector or the third feature vector of each category. The output layer outputs the category corresponding to the highest similarity value to obtain the prediction vector.

[0021] The predicted vector, the first feature vector, and the second feature vector are input into the contrastive loss layer for contrastive learning, and the loss value is output.

[0022] Optionally, the hate speech detection model is trained using the hate speech dataset, including:

[0023] The hate speech dataset is processed in batches to obtain several sample sets containing multiple positive and negative samples; wherein, the positive samples are classified as abnormal, and the negative samples are classified as normal.

[0024] The sample set is input into the hate speech detection model. In the contrast loss layer, cross-entropy is calculated between the predicted vector and the true label vector of the category to obtain a first loss value. The similarity between the first feature vector of each positive sample output by the contrast mapping layer and the second feature vector of other positive samples is calculated to obtain a second loss value. The similarity between the second feature vector of each positive sample output by the contrast mapping layer and the first feature vector of other positive samples is calculated to obtain a third loss value.

[0025] The first loss value, the second loss value, and the third loss value are summed to obtain the loss value;

[0026] When the loss value meets the preset training termination condition, the training is completed, and the trained hate speech detection model is obtained.

[0027] Optionally, the second feature vector and the third feature vector are obtained by the following method:

[0028] When the label name for the abnormal category is "extreme" and the label name for the normal category is "normal", the word embedding layer outputs two label vector sequences; wherein each label vector sequence consists of a first vector and a second vector.

[0029] The first and second vectors of the label vector with the label name "biased" are input into the semantic extraction layer and then average pooled to obtain the label vector with the label name "biased". The label vector is then input into the contrast mapping layer to obtain the second feature vector.

[0030] The first and second vectors of the label vector with the label name "normal" are input into the semantic extraction layer and then average pooled to obtain the label vector with the label name "normal". This label vector is then input into the contrast mapping layer to obtain the third feature vector.

[0031] Secondly, embodiments of the present invention also provide a hate speech detection device based on contrastive learning, comprising:

[0032] The data construction module is used to construct a dataset of hate speech based on preset sensitive keywords and a dialogue generation model;

[0033] The model building module is used to build a hate speech detection model and train the hate speech detection model using the hate speech dataset; wherein, the hate speech detection model includes a word embedding layer, a semantic extraction layer, a contrast mapping layer, a contrast loss layer, and an output layer in sequence;

[0034] The detection module is used to input the speech to be detected into the trained hate speech detection model and output the detection result of the speech to be detected.

[0035] Thirdly, embodiments of the present invention also provide a computing device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the hate speech detection method based on contrastive learning described above.

[0036] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the hate speech detection method based on contrastive learning described above.

[0037] Fifthly, embodiments of the present invention also provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the method described in any of the first aspects of this specification.

[0038] This invention provides a method, apparatus, and computer device for hate speech detection based on contrastive learning. The method first generates a hate speech dataset using preset sensitive keywords and a dialogue generation model. Then, it constructs a hate speech detection model based on a word embedding layer, a semantic extraction layer, a contrastive mapping layer, a contrastive loss layer, and an output layer. This model is trained using the hate speech dataset to obtain a trained hate speech detection model. By inputting the hate speech to be detected into this trained model, the detection result can be obtained. Thus, the negative samples in the hate speech dataset constructed by this invention also contain preset sensitive keywords, making the positive and negative samples more balanced and enhancing the diversity and quality of the dataset. Simultaneously, the introduction of a contrastive loss layer based on contrastive learning into the hate speech detection model further expands the semantic differences between positive and negative hate speech samples, improving the accuracy of hate speech detection. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of a hate speech detection method based on contrastive learning provided in an embodiment of the present invention;

[0041] Figure 2This is a schematic diagram of the architecture of a hate speech detection model provided in an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of the architecture of another hate speech detection model provided in an embodiment of the present invention;

[0043] Figure 4 This is a hardware architecture diagram of a computing device provided in an embodiment of the present invention;

[0044] Figure 5 This is a structural diagram of a hate speech detection device based on contrastive learning provided in an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0046] Existing methods for detecting hate speech fail to fully extract semantic information from short texts, and the specific manifestations and causes of the small difference between positive and negative samples of hate speech are as follows:

[0047] First, current deep learning models cannot effectively distinguish between positive and negative samples. This is because the vector sequences extracted by pre-trained models, after further semantic feature extraction via GRU or LSTM, often narrow the semantic feature distance between positive and negative samples. For example, the sentences "dislike violence" and "like violence," after having their word embedding matrices extracted by pre-trained models and then input into a recurrent neural network (RNN), a long short-term memory network (LSTM), and a gated recurrent unit (GRU), respectively, yield the final text semantic features. However, the semantic features of these two sentences are often very close in Euclidean distance, making it difficult for current hate speech models to effectively distinguish this type of text.

[0048] Secondly, traditional deep learning models typically treat labels as independent target variables, directly mapping them to categories, lacking a deep understanding of the complex relationship between labels and input text.

[0049] Therefore, based on the above reasons, the present invention provides a hate speech detection method based on contrastive learning.

[0050] The following is the concept of the present invention, such as Figure 1 As shown in the figure, this embodiment of the invention provides a hate speech detection method based on contrastive learning, the method comprising:

[0051] Step 100: Construct a dataset of hate speech based on preset sensitive keywords and a dialogue generation model;

[0052] Step 102: Construct a hate speech detection model and train it using a hate speech dialogue dataset; the hate speech detection model includes, in sequence, a word embedding layer, a semantic extraction layer, a contrast mapping layer, a contrast loss layer, and an output layer.

[0053] Step 104: Input the speech to be detected into the trained hate speech detection model and output the detection result of the speech to be detected.

[0054] In this embodiment of the invention, a hate speech dataset is first generated using preset sensitive keywords and a dialogue generation model. Then, a hate speech detection model is constructed based on a word embedding layer, a semantic extraction layer, a contrastive mapping layer, a contrastive loss layer, and an output layer. This model is trained using the hate speech dataset to obtain a trained hate speech detection model. By inputting the speech to be detected into this trained hate speech detection model, the detection result of the speech can be obtained. Thus, the negative samples of the hate speech dataset constructed in this invention also contain preset sensitive keywords, making the positive and negative samples more balanced and enhancing the diversity and quality of the dataset. Simultaneously, a contrastive loss layer based on contrastive learning is introduced into the hate speech detection model, further expanding the semantic differences between positive and negative hate speech samples and improving the accuracy of hate speech detection.

[0055] It should be noted that, in accordance with the requirements of the "Interim Measures for the Administration of Generative Artificial Intelligence Services" jointly issued by the State Internet Information Office and relevant departments, the definition of hate speech includes, but is not limited to, the following three points: (1) Strong emotional opposition: Hate speech often carries strong aggressive, insulting, or degrading emotions; (2) Semantic complexity: Hate speech includes direct insults, as well as expressing an unreasonable viewpoint through a pre-set malicious stance; (3) Context-dependent: The same sentence may have different emotional tones in different contexts. For example, "These people deserve to die" may be an expression of anger in a certain situation, but it may also be a clear hate statement against another group of people in some cases.

[0056] The following description Figure 1 The execution method of each step is shown.

[0057] In step 100, a hate speech dataset is constructed based on preset sensitive keywords and a dialogue generation model, including:

[0058] Obtain different types of hate speech as positive samples;

[0059] Input preset sensitive keywords into the dialogue generation model to generate negative samples;

[0060] A dataset of hate speech dialogues is constructed based on positive and negative samples.

[0061] It should be noted that the positive samples are collected from existing datasets widely used for large-scale language model evaluation, containing three categories of hate speech: profanity, prejudice, and illegal / criminal content. In this way, the positive samples cover different types of malicious speech, and the diversity and complexity of hate speech were fully considered during the sample annotation process, providing more comprehensive and representative positive samples for subsequent training. The hate speech dataset is also known as CHCD (Chinese Hostile Conversation Dataset), or simply the CHCD dataset.

[0062] In existing hate speech datasets, the negative samples (i.e., normal speech) are often too simplistic and cover only a limited range of topics, lacking in-depth discussion of sensitive issues. This leads to an imbalance between positive and negative samples, negatively impacting the training results of hate speech detection models. This invention improves the construction method of hate speech datasets by using preset sensitive keywords to express attitudes. A dialogue generation model ensures the text maintains semantic naturalness and consistency while preventing the infiltration of malicious speech. Thus, the generated negative samples containing preset sensitive keywords not only cover various common normal social scenarios but also effectively reflect the general values ​​of the public, enhancing the diversity and authenticity of the dataset. Furthermore, it reduces quality control issues arising from scraping negative samples from social media platforms.

[0063] In a preferred embodiment, the dialogue generation model adopts the ChatGLM4-9B model.

[0064] In this embodiment of the invention, the ChatGLM4-9B model has stronger inference performance, longer context processing capability, multilingual capability, multimodal capability, and efficient pre-training technology.

[0065] In step 102, a hate speech detection model is constructed, including:

[0066] The Roberta-wwm model is used as the word embedding layer; the MLP layer is used as the contrast mapping layer; and Bi-LSTM or Bi-GRU is used as the semantic extraction layer.

[0067] In this embodiment of the invention, the model uses the Roberta-wwn model, the Bi-LSTM model, and contrastive learning to gradually achieve automatic detection and classification of hate speech, resulting in an efficient and accurate hate speech detection model.

[0068] In this embodiment of the invention, the Roberta-wwm model achieves more efficient semantic modeling in Chinese contexts through whole-word masking technology, effectively avoiding the inaccuracies of Chinese word segmentation. Because the Roberta-wwm model can better capture the contextual information of words, especially when dealing with polysemous words, synonyms, and complex sentence structures, it can learn the contextual information of a large amount of Chinese text through pre-training in hate speech detection tasks. This ensures that the representation of each word not only includes the grammatical information of the word itself but also incorporates its semantics in a specific context. Through input text, the Roberta-wwm model can generate embedding vectors for each word. These embedding vectors form a highly distributed representation in the semantic space, reflecting the multidimensional semantic features of words.

[0069] In this embodiment of the invention, since both the Bidirectional Long Short-Term Memory Network (Bi-LSTM) and the Bidirectional Gated Recurrent Unit (Bi-GRU) employ a bidirectional structure, they can process the input sequence from front to back or from back to front, thereby simultaneously capturing both the preceding and following contextual information in the input sequence. Therefore, applying Bi-LSTM or Bi-GRU to hate speech detection tasks can effectively capture these semantic nuances through their bidirectional contextual modeling capabilities, improving the understanding of complex text and ultimately enhancing the detection accuracy of the hate speech detection model.

[0070] In this embodiment of the invention, contrastive learning can construct positive and negative sample pairs, enabling the model to learn how to narrow the distance between similar samples in the feature space while widening the distance between dissimilar samples, thereby effectively distinguishing between positive and negative samples and thus effectively improving the discrimination and generalization capabilities of the hate speech detection model.

[0071] In a preferred embodiment, construct as follows Figure 2 The hate speech detection model shown includes:

[0072] The input text is obtained by appending the label information of each category to the text to be entered; the categories include abnormal and normal.

[0073] Input the text into the word embedding layer and output the context vector, the label vector sequence of each category, and the word embedding vector matrix.

[0074] The label vector sequences and word embedding vector matrices of each category are input into the semantic extraction layer, which outputs several semantic feature vectors; the semantic feature vectors include the label vectors output from the label vectors of each category;

[0075] The context vector, the label vector of the abnormal category, and the label vector of the normal category are input into the contrast mapping layer, and the first feature vector, the second feature vector, and the third feature vector are output respectively.

[0076] The first feature vector is compared with the second or third feature vector of each category. The output layer outputs the category corresponding to the highest similarity value to obtain the prediction vector.

[0077] The predicted vector, the first feature vector, and the second feature vector are input into the contrastive loss layer for contrastive learning, and the loss value is output.

[0078] It should be noted that in this embodiment of the invention, the label name for the abnormal category is "extreme," meaning the label information is "extreme"; the label name for the normal category is "normal," meaning the label information is "normal." Specifically, the context vector integrates context information and label information; the label vector is obtained by converting the label information into vector form, resulting in two label vectors; the input text is converted into vector form in the word embedding layer of the feature space to obtain a word embedding vector matrix, where each vector in the matrix corresponds one-to-one with each character in the input text, meaning the number of vectors in the matrix is ​​the same as the number of characters in the input text.

[0079] For the label vector, for example, if the label information is "hate" and consists of a single character, the word embedding layer will output a vector, which is a sequence of label vectors for the abnormal category. For example, if the label information is "extreme" and consists of two characters, the word embedding layer will output two vectors. In this case, the two vectors need to be merged (i.e., average pooling) to obtain a one-dimensional vector, which is a sequence of label vectors for the abnormal category.

[0080] Specifically, in the contrast mapping layer, the context vector is input into the contrast mapping layer and the first feature vector is output; the label vector of the abnormal category is input into the contrast mapping layer and the second feature vector is output; the label vector of the normal category is input into the contrast mapping layer and the third feature vector is output.

[0081] In this embodiment of the invention, by concatenating label information to the beginning of the input text and then inputting the text into the word embedding layer, the word embedding form of the text is obtained, and the context vector, the label vector sequence of each category included in the hate speech dataset, and the word embedding vector matrix are output. This utilizes the features of the label information and the CLS features of the word embedding layer to widen the distance between positive and negative samples. Simultaneously, in the semantic extraction layer, a deeper understanding of the complex relationship between the label information and the input text is achieved, improving the comprehension ability. Therefore, this invention enhances the accuracy of hate speech text classification by fully utilizing label information.

[0082] In this embodiment of the invention, multiple experiments have demonstrated that the contrast mapping layer improves the learning performance of the hate speech detection model. By inputting the context vector and semantic feature vector into the contrast mapping layer MLP layer respectively, the feature vector is mapped to an optimized feature space, thereby promoting the objective function of contrastive learning to learn and optimize more effectively.

[0083] In step 102, the hate speech detection model is trained using a hate speech dataset, including:

[0084] The hate speech dataset was processed in batches to obtain several sample sets containing positive and negative samples; the positive samples were classified as abnormal, and the negative samples were classified as normal.

[0085] The sample set is input into the hate speech detection model. In the contrast loss layer, the cross-entropy of the predicted vector and the true label vector of the category is calculated to obtain the first loss value. The similarity between the first feature vector of each positive sample output by the contrast mapping layer and the second feature vector of other positive samples is calculated to obtain the second loss value. The similarity between the second feature vector of each positive sample output by the contrast mapping layer and the first feature vector of other positive samples is calculated to obtain the third loss value.

[0086] The first loss value, the second loss value, and the third loss value are summed to obtain the total loss value.

[0087] Training is completed when the loss value meets the preset training termination condition, and a trained hate speech detection model is obtained.

[0088] In this embodiment of the invention, in order to further expand the spatial distance between hate speech (i.e., positive samples) and normal speech (i.e., negative samples) in the feature space, the hate speech dataset is processed in batches to construct a sample set containing positive and negative samples.

[0089] Specifically, the loss function in the contrast loss layer consists of three parts. The first loss value represents the cross-entropy loss between the model's predicted vector and the true label vector of the class (i.e., in one-hot form). Loss CE It can be obtained through the following formula:

[0090] (1)

[0091] like Figure 3 As shown, the second loss value represents the first feature vector of each positive sample. e CLS The semantic vector of the label of similar samples (i.e., the second feature vector). e POS Losses between Loss CLS It can be obtained through the following formula:

[0092] (2)

[0093] In formula (2), the first feature vector of each positive sample e CLS Second feature vectors of other similar samples e POS Perform a dot product, then divide by the temperature parameter to scale the dot product value to obtain the similarity score, and finally obtain the similarity matrix. logits 1. Among them, the similarity matrix logits In Figure 1, the value in each row represents the similarity score of the sample to other samples of the same type. Then, the similarity matrix is ​​analyzed. logits 1. Normalize and sum the values ​​in each row to obtain the total similarity of each sample to other samples of the same class.

[0094] like Figure 3 As shown, the third loss value represents the second feature vector of each positive sample. e POS The first feature vector of the same type of sample e CLS Losses between Loss TAG It can be obtained through the following formula:

[0095] (3)

[0096] The calculation process of formula (3) is similar to that of formula (2), wherein, in the above formulas (1) to (3), wherein, P i For the first i The set of positive samples in a sample set; label p for P i A positive sample in the sample; context i The first feature vector of the target samplee CLS ; A i For the first i The set of positive and negative samples in a sample set; I This is the set of all samples in the hate speech dataset. N The total number of samples; K For a set of category labels, k For one of the category labels; τ For temperature parameters;

[0097] Then the loss value Loss = Loss CE + Loss CLS + Loss TAG .

[0098] It should be noted that, Figure 3 The output layer is not shown, and the semantic extraction layer only shows one semantic feature vector; and when the input is a negative sample, the output of the comparison mapping layer is... e NEG This represents the third feature vector; when the input is a positive sample, it is compared with the output of the mapping layer. e POS This represents the second eigenvector.

[0099] In this embodiment of the invention, the loss function of the hate speech detection model, through the above three parts, narrows the distance between positive samples and effectively expands the distance between positive and negative samples, enabling the hate speech detection model to learn more detailed features, thereby improving its ability to classify unknown texts.

[0100] In a preferred embodiment, the second feature vector and the third feature vector are obtained by the following method:

[0101] When the label name for the abnormal category is "extreme" and the label name for the normal category is "normal", the word embedding layer outputs two label vector sequences; each label vector sequence consists of a first vector and a second vector.

[0102] The first and second vectors of the label vector with the label name "biased" are input into the semantic extraction layer and then average pooled to obtain the label vector with the label name "biased". This label vector is then input into the contrast mapping layer to obtain the second feature vector.

[0103] The first and second vectors, labeled as normal, are input into the semantic extraction layer and then average pooled to obtain the label vector with normal label name; this label vector is then input into the contrast mapping layer to obtain the third feature vector.

[0104] Since the tag names are in Chinese and a tag may consist of multiple characters, average pooling is required to obtain the word embedding sequence of each tag. In the dataset of the present invention, there are two categories of labels. One tag name is "extreme", and the other tag name is "normal". Taking "extreme" as an example, inputting "extreme" into the word embedding layer outputs a tag vector sequence including the first vector V1 and the second vector V2. After passing through the semantic extraction layer and performing average pooling, the tag vector V is obtained. p The tag vector V p is output as the second feature vector after passing through the contrast mapping layer. e POS The method for obtaining the third feature vector is the same.

[0105] In the embodiment of the present invention, starting from the conversation remarks, preset sensitive keywords are introduced to construct a dataset of hateful conversation remarks for the user to dialogue with the large model. At the same time, a hateful speech detection model is built based on Bi-LSTM or Bi-GRU, effectively mining the deep semantic information in the text, and using the pre-trained model Roberta-wwm and contrast learning technology to expand the distance between positive and negative samples, effectively refining the semantic information between hateful speech and non-hateful speech, so as to effectively detect the hateful speech in the user's conversation and improve the accuracy of hateful speech detection.

[0106] After actual testing, the mainstream model and the hateful speech detection model of the present invention are used to detect hateful speech, and the test results of the detection accuracy rate (%) as shown in Table 1 and Table 2 are obtained. Among them, Table 1 is trained using the existing commonly used hateful speech dataset, and Table 2 is trained using the hateful conversation remarks dataset constructed in the above step 100. Among them, the topics of the two datasets are the same, and the ratio of positive and negative samples is the same. However, the negative sample content in the dataset used in Table 1 is too simple, and the sensitive topics involved are also relatively single.

[0107] Table 1

[0108]

[0109] Table 2

[0110]

[0111] It should be noted that in Table 1, in the first row, when the existing model includes word embedding layers, FNN, and uses cross-entropy in the loss function, the accuracy on the validation dataset is 92.02%, and the accuracy on the test dataset is 82.4%; when the existing model includes word embedding layers, Bi-GRU, and uses cross-entropy in the loss function, the accuracy on the validation dataset is 91.88%, and the accuracy on the test dataset is 82.25%; when the existing model includes word embedding layers, Bi-LSTM, and uses cross-entropy in the loss function, the accuracy on the validation dataset is 92.07%, and the accuracy on the test dataset is 82.27%. In the second row, the existing model architecture, including word embedding layers, FNN, and cross-entropy + basic contrastive learning in the loss function, achieves an accuracy of 92.4% on the validation dataset and 82.74% on the test dataset. The existing model architecture, including word embedding layers, Bi-GRU, and cross-entropy + basic contrastive learning in the loss function, achieves an accuracy of 92.07% on the validation dataset and 82.4% on the test dataset. The existing model architecture, including word embedding layers, Bi-LSTM, and cross-entropy + basic contrastive learning in the loss function, achieves an accuracy of 92.32% on the validation dataset and 82.83% on the test dataset. It should be noted that in the existing models in the first and second rows, the input is the text to be input.

[0112] However, the existing models described above still have relatively low accuracy in detecting hate speech. Therefore, to improve these existing models, in the third row, experimental model A was designed, which includes a word embedding layer, a contrast mapping layer (MLP layer), an FNN, and uses cross-entropy in the loss function combined with the contrast learning framework of this invention. Its accuracy on the validation dataset is 92.49%, and its accuracy on the test dataset is 82.83%. Experimental model B was also designed, which includes a word embedding layer, a semantic extraction layer (Bi-GRU), a contrast mapping layer (MLP layer), and uses cross-entropy in the loss function combined with the contrast learning framework of this invention. Its accuracy on the validation dataset is 92.41%, and its accuracy on the test dataset is 82.96%. Furthermore, experimental model C was designed, which includes a word embedding layer, a semantic extraction layer (Bi-LSTM), a contrast mapping layer (MLP layer), and uses cross-entropy in the loss function combined with the contrast learning framework of this invention. Its accuracy on the validation dataset is 92.29%, and its accuracy on the test dataset is 83.13%. Clearly, experimental models B and C provided in this embodiment of the invention have higher accuracy in detecting hate speech. In the model in the third row, the input is text to be input, which is concatenated with labels of various categories. As shown in Table 2, after training with the hate speech dataset constructed in this invention, the detection accuracy of experimental models B and C provided in this embodiment of the invention has been further improved, with an accuracy of 95.05% on the validation dataset and 94.98% on the test dataset.

[0113] As shown in Tables 1 and 2, the hate speech detection model of this invention has a higher detection accuracy than mainstream models. The hate speech detection method based on contrastive learning proposed in this invention has the following advantages:

[0114] (1) By using contrastive learning, the model can learn more detailed features by comparing the similarity between positive and negative samples, thereby improving the ability to classify unseen samples. In the hate speech detection task, although the surface features of hate speech and normal speech may be similar, through contrastive learning, the model can better distinguish subtle semantic differences, thereby improving the detection accuracy. The effect is about 2% better than the mainstream model.

[0115] (2) The hate speech dataset contains diverse positive samples of hate speech. This invention effectively expands the types and forms of hate speech by collecting positive samples of three categories of hate speech: profanity, prejudice, discrimination, and illegal and criminal activities. These positive samples not only have strong offensiveness and inflammatory nature, but also comprehensively reflect the complexity of hate speech, ensuring that the training data is richer and more diverse.

[0116] (3) Negative samples are generated based on preset sensitive keywords and dialogue generation model, which effectively avoids quality control problems in crawling data and ensures the high quality and consistency of negative sample content.

[0117] like Figure 4 , Figure 5 As shown, this embodiment of the invention provides a hate speech detection device based on contrastive learning. The device embodiment can be implemented through software, hardware, or a combination of both. From a hardware perspective, as... Figure 4 The diagram shown is a hardware architecture diagram of a computing device housing a hate speech detection device based on contrastive learning, as provided in an embodiment of the present invention. (Except for...) Figure 4 In addition to the processor, memory, network interface, and non-volatile memory shown, the computing device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing packets. Taking software implementation as an example, such as... Figure 5 As shown, a device in a logical sense is formed by the CPU of its computing device reading the corresponding computer program from non-volatile memory into memory and running it. This embodiment provides a hate speech detection device based on contrastive learning, comprising:

[0118] Data construction module 500 is used to construct a dataset of hate speech based on preset sensitive keywords and a dialogue generation model;

[0119] The model building module 502 is used to build a hate speech detection model and train the hate speech detection model using a hate speech dataset; wherein, the hate speech detection model includes a word embedding layer, a semantic extraction layer, a contrast mapping layer, a contrast loss layer, and an output layer in sequence.

[0120] The detection module 504 is used to input the speech to be detected into the trained hate speech detection model and output the detection result of the speech to be detected.

[0121] In some specific implementations, the data construction module 500 can be used to perform the above step 100, the model construction module 502 can be used to perform the above step 102, and the detection module 504 can be used to perform the above step 104.

[0122] In some specific implementations, the data construction module 500 is also used to perform the following operations:

[0123] Obtain different types of hate speech as positive samples;

[0124] Input preset sensitive keywords into the dialogue generation model to generate negative samples;

[0125] A dataset of hate speech dialogues is constructed based on positive and negative samples.

[0126] In some specific implementations, the model building module 502 is also used to perform the following operations:

[0127] The Roberta-wwm model is used as the word embedding layer; the MLP layer is used as the contrast mapping layer; and Bi-LSTM or Bi-GRU is used as the semantic extraction layer.

[0128] In some specific implementations, the model building module 502 is also used to perform the following operations:

[0129] The input text is obtained by appending the label information of each category to the text to be entered; the categories include abnormal and normal.

[0130] Input the text into the word embedding layer and output the context vector, the label vector sequence of each category, and the word embedding vector matrix.

[0131] The label vector sequences and word embedding vector matrices of each category are input into the semantic extraction layer, which outputs several semantic feature vectors; the semantic feature vectors include the label vectors output from the label vectors of each category;

[0132] The context vector, the label vector of the abnormal category, and the label vector of the normal category are input into the contrast mapping layer, and the first feature vector, the second feature vector, and the third feature vector are output respectively.

[0133] The first feature vector is compared with the second or third feature vector of each category. The output layer outputs the category corresponding to the highest similarity value to obtain the prediction vector.

[0134] The predicted vector, the first feature vector, and the second feature vector are input into the contrastive loss layer for contrastive learning, and the loss value is output.

[0135] In some specific implementations, the model building module 502 is also used to perform the following operations:

[0136] The hate speech dataset was processed in batches to obtain several sample sets containing positive and negative samples; the positive samples were classified as abnormal, and the negative samples were classified as normal.

[0137] The sample set is input into the hate speech detection model. In the contrast loss layer, the cross-entropy of the predicted vector and the true label vector of the category is calculated to obtain the first loss value. The similarity between the first feature vector of each positive sample output by the contrast mapping layer and the second feature vector of other positive samples is calculated to obtain the second loss value. The similarity between the second feature vector of each positive sample output by the contrast mapping layer and the first feature vector of other positive samples is calculated to obtain the third loss value.

[0138] The first loss value, the second loss value, and the third loss value are summed to obtain the total loss value.

[0139] Training is completed when the loss value meets the preset training termination condition, and a trained hate speech detection model is obtained.

[0140] In some specific implementations, the model building module 502 is also used to perform the following operations:

[0141] When the label name for the abnormal category is "extreme" and the label name for the normal category is "normal", the word embedding layer outputs two label vector sequences; each label vector sequence consists of a first vector and a second vector.

[0142] The first and second vectors of the label vector with the label name "biased" are input into the semantic extraction layer and then average pooled to obtain the label vector with the label name "biased". This label vector is then input into the contrast mapping layer to obtain the second feature vector.

[0143] The first and second vectors, labeled as normal, are input into the semantic extraction layer and then average pooled to obtain the label vector with normal label name; this label vector is then input into the contrast mapping layer to obtain the third feature vector.

[0144] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on a hate speech detection device based on contrastive learning. In other embodiments of the present invention, a hate speech detection device based on contrastive learning may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0145] The information interaction and execution process between the modules in the above-mentioned device are based on the same concept as the method embodiment of the present invention, and the specific details can be found in the description of the method embodiment of the present invention, and will not be repeated here.

[0146] This invention also provides a computing device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements a hate speech detection method based on contrastive learning according to any embodiment of this invention.

[0147] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform a hate speech detection method based on contrastive learning according to any embodiment of this invention.

[0148] Embodiments of this application also provide a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium and executes the computer program, causing the computer device to perform any of the contrastive learning-based hate speech detection methods described in the above embodiments.

[0149] Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer (or CPU or MPU) of the system or apparatus may read and execute the program code stored in the storage medium.

[0150] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0151] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0152] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, system, or device.

[0153] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0154] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0155] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0156] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion module connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion module execute some and all of the actual operations, thereby realizing the function of any of the above embodiments.

[0157] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0158] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hate speech detection method based on contrastive learning, characterized in that, include: A dataset of hate speech dialogues was constructed based on preset sensitive keywords and a dialogue generation model. A hate speech detection model is constructed and trained using the hate speech dataset. The hate speech detection model includes, in sequence, a word embedding layer, a semantic extraction layer, a contrast mapping layer, a contrast loss layer, and an output layer. The Roberta-WWM model is used as the word embedding layer; an MLP layer is used as the contrast mapping layer; and Bi-LSTM or Bi-GRU is used as the semantic extraction layer. The target speech is input into the trained hate speech detection model, and the detection result of the target speech is output. The construction of the hate speech detection model includes: The input text is obtained by concatenating the label information of each category before the text to be input; wherein, the categories include abnormal and normal. The input text is input into the word embedding layer, and the output is a context vector, a sequence of label vectors for each category, and a word embedding vector matrix; The label vector sequence of each category and the word embedding vector matrix are input into the semantic extraction layer, and several semantic feature vectors are output; the semantic feature vectors include label vectors output from the label vectors of each category; The context vector, the label vector of the abnormal category, and the label vector of the normal category are input into the contrast mapping layer, and the first feature vector, the second feature vector, and the third feature vector are output respectively. The first feature vector is compared with the second feature vector or the third feature vector of each category. The output layer outputs the category corresponding to the highest similarity value to obtain the prediction vector. The predicted vector, the first feature vector, and the second feature vector are input into the contrastive loss layer for contrastive learning, and a loss value is output. The loss function contained in the contrastive loss layer consists of three parts: the first loss value represents the cross-entropy loss between the predicted vector of the model and the true label vector of the class; the second loss value represents the loss between the first feature vector of each positive sample and the second feature vector of the same class sample; and the third loss value represents the loss between the second feature vector of each positive sample and the first feature vector of the same class sample.

2. The method according to claim 1, characterized in that, The dataset of hate speech constructed based on preset sensitive keywords and a dialogue generation model includes: Obtain different types of hate speech as positive samples; Input preset sensitive keywords into the dialogue generation model to generate negative samples; Based on the positive and negative samples, the hate speech dataset is constructed.

3. The method according to claim 1, characterized in that, Training the hate speech detection model using the aforementioned hate speech dataset includes: The hate speech dataset is processed in batches to obtain several sample sets containing positive and negative samples; wherein, the positive samples are classified as abnormal, and the negative samples are classified as normal. The sample set is input into the hate speech detection model. In the contrast loss layer, cross-entropy is calculated between the predicted vector and the true label vector of the category to obtain a first loss value. The similarity between the first feature vector of each positive sample output by the contrast mapping layer and the second feature vector of other positive samples is calculated to obtain a second loss value. The similarity between the second feature vector of each positive sample output by the contrast mapping layer and the first feature vector of other positive samples is calculated to obtain a third loss value. The first loss value, the second loss value, and the third loss value are summed to obtain the loss value; When the loss value meets the preset training termination condition, the training is completed, and the trained hate speech detection model is obtained.

4. The method according to claim 1, characterized in that, The second feature vector and the third feature vector are obtained by the following method: When the label name for the abnormal category is "extreme" and the label name for the normal category is "normal", the word embedding layer outputs two label vector sequences; wherein each label vector sequence consists of a first vector and a second vector. The first and second vectors of the label vector with the label name "biased" are input into the semantic extraction layer and then average pooled to obtain the label vector with the label name "biased". The label vector is then input into the contrast mapping layer to obtain the second feature vector. The first and second vectors of the label vector with the label name "normal" are input into the semantic extraction layer and then average pooled to obtain the label vector with the label name "normal". This label vector is then input into the contrast mapping layer to obtain the third feature vector.

5. A hate speech detection device based on contrastive learning, used to implement the method described in any one of claims 1 to 4, characterized in that, include: The data construction module is used to construct a dataset of hate speech based on preset sensitive keywords and a dialogue generation model; The model building module is used to build a hate speech detection model and train the hate speech detection model using the hate speech dataset; wherein, the hate speech detection model includes a word embedding layer, a semantic extraction layer, a contrast mapping layer, a contrast loss layer, and an output layer in sequence; The detection module is used to input the speech to be detected into the trained hate speech detection model and output the detection result of the speech to be detected.

6. A computing device comprising a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program, implements the method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-4.

8. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-4.

Images