Hatred recognition method combining multi-layer hatred generation and progressive comparative learning
Through the method of multi-level hatred generation and gradual comparative learning, the problem of obscure and confrontational hatred speech recognition in the existing technology is solved, and comprehensive coverage and robust recognition of hatred expressions at different levels are achieved.
Patent Information
- Application Number
- CN202510658260.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-21
- Publication Date
- 2025-08-15
AI Technical Summary
The prior art is difficult to effectively identify obscure and confrontational hate speech, and lacks data generation capabilities and progressive learning strategies, resulting in insufficient detection robustness.
The multi-level hatred generation and progressive contrast learning method is adopted, and the model is gradually optimized to identify explicit, implicit and confrontational hatred speech through feature coding, multi-level hatred generation and feature extraction modules. The BERT model and HateBERT are used for feature processing and scoring, and the model is trained in stages with progressive contrast learning.
It significantly improves the ability to identify obscure and confrontational hate speech, improves the robustness of the model and the ability to adapt to complex scenarios, and accurately recognizes various hatred expressions.
Smart Images
Figure CN120492628A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of natural language processing and artificial intelligence, and relates to a hatred recognition method that combines multi-level hatred generation with progressive contrast learning. Background Art
[0002] With the rapid development of the Internet and social media, while people are free to express their opinions on online platforms, the spread of hate speech has become increasingly serious. Hate speech refers to discriminatory, inflammatory or offensive information directed at specific groups or individuals through insults, attacks, defamation, etc. These speeches not only hurt the feelings of individuals or groups, but may also intensify social conflicts and threaten public order. Existing technologies for hate speech detection are mainly based on two methods: (1) Traditional methods based on keyword matching: text is compared and identified through a pre-set sensitive vocabulary library. This method has a good detection effect on explicit hate speech, but it is difficult to identify implicit expressions and adversarial samples. For example, when insulting language is transformed into implicit expressions through spelling deformation, metaphors or rewriting; (2) Intelligent detection methods based on machine learning and deep learning: using neural network models (such as LSTM, Transformer, BERT, etc.) to model text semantics, it can understand complex semantics to a certain extent and identify hate speech.
[0003] However, the performance of these methods is highly dependent on the quality and coverage of the training data. When the dataset cannot cover implicit hate speech or adversarial examples, the detection effect is significantly reduced. Implicit hate speech is a complex form of offensive expression that expresses aggression covertly through metaphors, irony, or puns, which increases the difficulty of detection. Its main challenges include: (1) semantic concealment: the hateful intention is hidden deep in the language and appears neutral or harmless on the surface, making it difficult for traditional methods to identify; (2) context dependence: the context is needed to understand the potential offensive intention in the language; (3) diversity of expression forms: implicit expressions are diverse and difficult to capture through fixed patterns or single features. In addition, the emergence of adversarial examples further exacerbates the challenges of hate speech detection. Adversarial examples are texts generated by modifying the original attack text (such as synonym replacement, spelling perturbation, or sentence transformation) in order to circumvent the recognition rules of the detection system. This strategy greatly reduces the robustness of existing detection systems.
[0004] In response to the above problems, existing technologies have the following deficiencies: (1) Insufficient data generation capabilities: Existing data augmentation technologies mostly focus on the generation of explicit hate speech and lack the ability to generate implicit and adversarial samples, resulting in insufficient diversity of training data; (2) Single detection strategy: Traditional detection methods mainly target explicit hate speech and have poor adaptability to implicit hate expressions; (3) Lack of gradual optimization mechanism: Existing methods usually directly train models without progressive learning strategies, making it difficult for models to gradually adapt to complex scenarios. Summary of the Invention
[0005] To address the above-mentioned problems in the prior art, the present invention adopts a hate recognition method that combines multi-level hate generation with progressive contrastive learning, including: obtaining speech, inputting the speech into a trained implicit hate recognition model to obtain speech features, and obtaining hate recognition results based on the speech features; the implicit hate recognition model includes: a feature encoding module, a multi-level hate generation module, and a feature extraction module;
[0006] The training process of the implicit hate detection model includes:
[0007] S1. Obtain a speech set, which includes normal speech and hate speech; i Input feature encoding module to obtain speech encoding feature f i ; Speech coding features include: hate speech coding features f i h and normal speech encoding features f i n ; Where i represents the index of the speech;
[0008] S2. Encode hate speech features f i h Input the multi-level hate generation module to get multi-level hate speech
[0009] S3. Multi-layered hate speech Input the feature extraction module to obtain the multi-level hate speech feature F i h ;
[0010] S4. Based on the multi-level hate speech characteristics F i h and speech encoding features f i Calculate the total loss function value, and update the parameters of the implicit hatred recognition model according to the total loss function value. When the loss function value is minimized, the trained implicit hatred recognition model is obtained.
[0011] Beneficial effects:
[0012] 1. This invention significantly improves the diversity and coverage of training data through a multi-level hate generation module. From explicit attacks to covert attacks and disguised attacks, it comprehensively covers different levels of hate expression, enabling accurate identification of various expressions, especially the offensive intent in implicit expressions, thereby improving the robustness of recognition. 2. This invention adopts a progressive contrastive learning strategy, dividing the model training process into three stages: explicit hate detection, implicit hate detection, and adversarial hate detection. This gradually optimizes the model's feature extraction capabilities, allowing the model to gradually adapt to complex scenarios and improving the robustness of recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 A schematic diagram of a flow chart of a method for identifying hatred that combines multi-level hatred generation with progressive contrastive learning, provided by an embodiment of the present invention;
[0014] Figure 2 A structural diagram of a hate recognition method combining multi-level hate generation and progressive contrastive learning provided by an embodiment of the present invention;
[0015] Figure 3 This is a network structure diagram of the multi-level hatred generation module provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0016] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0017] like Figure 1 、 Figure 2 As shown, the present invention adopts a hate recognition method that combines multi-level hate generation and progressive contrastive learning, including: obtaining speech, inputting the speech into a trained implicit hate recognition model to obtain speech features, and predicting hate recognition results based on the speech features; predicting hate recognition results based on speech features includes: processing the speech features using a softmax function to obtain predicted scores of hate and non-hatred, and the result corresponding to the maximum predicted score is the hate recognition result.
[0018] The implicit hate recognition model includes a feature encoding module, a multi-level hate generation module, and a feature extraction module. The training process of the implicit hate recognition model includes:
[0019] S1. Obtain speech set D, which includes normal speech and hate speech; i Input feature encoding module to obtain speech encoding feature fi ; Speech coding features include: hate speech coding features f i h and normal speech encoding features f i n ; Where i represents the index of the speech, and the feature encoding module is the BERT model;
[0020] S2. Encode hate speech features f i h Input the multi-level hate generation module to get multi-level hate speech
[0021] The multi-level hatred generation module includes: explicit generation module, implicit generation module and adversarial generation module; Figure 3 As shown, the multi-level hate generation module encodes the hate speech features f i h Processing includes: encoding hate speech features f i h Input the explicit generation module, implicit generation module and adversarial generation module respectively to obtain multi-level hate speech Among them, multi-level hate speech Including the best explicit hate speech implicit hate speech and confrontational hate speech
[0022] The core of explicit hate speech is direct aggression. Therefore, the screening method of the Explicit Generation Module (EGM) mainly focuses on: offensive intensity and semantic consistency. Among them, offensive intensity measures the number and intensity of directly offensive words in the sample, and semantic consistency measures whether the generated sample is semantically consistent with the original hate sample.
[0023] Specifically, the explicit generation module encodes the hate speech feature f i h Processing includes:
[0024] Encoding hate speech features f i h Input the pre-trained large language model (such as GPT, BERT) to obtain the explicit hatred generation sample set S i ={s i,1 ,s i,2 ,…,s i,N}; where s ij Generate a sample for the jth explicit hatred, and N is the number of samples generated for explicit hatred;
[0025] Obtain the existing Hate Speech and Offensive Language (HSOL) database, and generate samples s for each explicit hate word based on the database. ij The hate words in each explicit hate generation sample are weighted and summed to obtain the aggressiveness of each explicit hate generation sample. The specific calculation formula is as follows:
[0026]
[0027] Among them, Aggressiveness(s ij ) is sample s ij The aggressiveness of sample s ij The vocabulary in Is to judge whether the word ω is in the hate word library HSOL, if so, the word ω is a hate word and is 1, otherwise the word ω is not a hate word and is 0; Weight(ω) is the weight of the word ω in HSOL.
[0028] Calculate each generated sample s ij With the original speech x i The semantic consistency of each explicit hatred generation sample is obtained by the following calculation formula:
[0029] Similarity(x i ,s ij )=cos(x i ,s ij )(2)
[0030] Among them, Similarity(O,s ij ) is sample s ij With hate speech coding features f i h The corresponding original speech x i The semantic consistency score of cos(x i ,s ij ) is sample s ij With the original speech x i The cosine similarity of .
[0031] Each generated sample s ij The offensive strength of each generated sample s is integrated with the semantic consistency score. ij Total score Score(s ij ), calculated as follows:
[0032] Score(s ij )=Aggressiveness(s ij )+Similarity(x i ,s ij ) (3)
[0033] Select the explicit hate-generating sample with the highest total score as the optimal explicit hate-generating speech
[0034] The characteristic of implicit hate speech is the indirectness of aggressive expression. Therefore, the implicit generation module (IGM) mainly focuses on: metaphor and concealment, moderate aggressiveness, and semantic consistency. Among them, metaphor and concealment measure whether the implicit hate generation sample expresses aggressive intentions through non-offensive words, moderate aggressiveness measures whether the implicit hate generation sample retains potential aggressiveness, and semantic consistency measures whether the implicit hate generation sample is semantically consistent with the original hate sample.
[0035] Specifically, the implicit generation module encodes the hate speech features f i h Processing includes:
[0036] Encoding hate speech features f i h Input the pre-trained large language model (such as GPT, BERT) to obtain the implicit hatred generation sample set H i ={h i1 ,h i2 ,…,h iN}; where h ij Generate a sample for the jth implicit hatred, where N is the number of implicit hatred samples; these samples appear neutral on the surface but actually contain offensive content.
[0037] Obtain a hate vocabulary library and calculate the implicit hate generation sample h based on the hate vocabulary library ij The proportion of non-offensive words in the sentence is used to obtain the metaphor score; the metaphor score reflects the degree of obscurity of the offensive expression, and the calculation formula is as follows:
[0038]
[0039] Among them, IImplicit(h ij ) Generate sample h for implicit hatred ij Metaphorical rating of Is to judge whether the word ω is in the hate word library HSOL, len(h ij ) is the sample h ij The length of includes the total number of words.
[0040] Calculate implicit hatred generation sample h through VADER ij The potential offensiveness score (negative sentiment intensity) of the text is obtained by VADER (Valence Aware Dictionary and sEntiment Reasoner), which is a tool specifically used to analyze text sentiment and can provide positive, neutral, and negative sentiment intensity.
[0041] Calculate the implicit hatred generation sample h ij With hate speech coding features f i h The corresponding original speech x i The semantic consistency score is calculated in the same way as the semantic consistency in EGM.
[0042] The metaphor score, potential offensive score and semantic consistency score are integrated to obtain the total score Score (h ij ), the formula is as follows:
[0043]
[0044] Among them, Latent_Aggressiveness(h ij ) is the sample h ij The potential aggressiveness score, Similarity(h ij ) is the sample h ij With the original speech x i Semantic consistency score.
[0045] Generate the implicit hatred sample h with the highest total score ij As the optimal implicit hate speech
[0046] Adversarial hate speech examples are characterized by being difficult to identify as hate speech on the surface, yet inherently offensive. The goal of adversarial hate speech examples is to deceive the model. While difficult to identify as hate speech on the surface, they still contain potentially aggressive content. Therefore, the Adversarial Generation Module (AGM) must prioritize both adversarial and concealing capabilities, while also maintaining semantic consistency.
[0047] Adversarial generation module encodes features f of hate speech i h Processing includes:
[0048] According to the hate speech coding feature f i h Generate adversarial hatred generation sample set A using pre-trained large language model i ={a i1 ,a i2,…,a iN}; where a ij Generate samples for the jth adversarial hatred, N is the number of adversarial hatred generated samples; these samples avoid being detected by changing their surface form.
[0049] The existing pre-trained hate speech detection model HateBERT is used to predict the probability of each adversarial hate generation sample being hate speech, and the probability of each adversarial hate generation sample a is calculated based on the probability of hate speech. ij The adversarial score is calculated as follows:
[0050] Adversarial_Score(a ij )=1-Hate_Rate(a ij ) (6)
[0051] Among them, Aversarial_Score(a ij ) is sample a ij The adversarial score, Hate_Rate(a ij ) is the prediction sample a for the hate detection model HateBERT ij The probability of hate speech.
[0052] The calculation of metaphoricality and semantic consistency scores is consistent with that in IGM.
[0053] Integrate the adversarial score, metaphor score and semantic consistency score to get sample a ij Total score of Score(a ij ), as shown in formula (7):
[0054] Score(a ij )=Adversarial_Score(a ij )+Implicity(a ij )+Similarity(a ij ,x i ) (7)
[0055] Among them, Implicity (a ij ) is sample a ij The implicitness score, Similarity(a ij ,x i ) is sample a ij With hate speech coding features f i h The corresponding original speech x i Semantic consistency score.
[0056] The adversarial generated sample with the highest total score is taken as the optimal adversarial hate speech generation
[0057] S3. Multi-layered hate speech Input the feature extraction module to obtain the multi-level hate speech feature F i h ;
[0058] The feature extraction module is a Transform network, which includes twelve network layers; among them, the 1st to 4th network layers are the bottom group, the 5th to 8th network layers are the middle group, and the 9th to 12th network layers are the high-level group.
[0059] Multi-layered characteristics of hate speech in, Represents the multi-level hate speech features output by the bottom group, middle group, and high-level group of the feature extraction module; the multi-level hate speech features of each group c Includes: explicit characteristics of hate speech Characteristics of implicit hate speech and characteristics of confrontational hate speech
[0060] S4. Based on the multi-level hate speech characteristics F i h and speech encoding features f i Calculate the total loss function value, and update the parameters of the implicit hatred recognition model according to the total loss function value. When the loss function value is minimized, the trained implicit hatred recognition model is obtained.
[0061] The present invention adopts a phased training strategy through progressive contrastive learning, and trains explicit, implicit and adversarial hate speech in sequence according to the complexity of the samples. Figure 2 The pipeline of progressive contrastive learning is presented, which gradually improves the performance of the model by optimizing the loss function in stages.
[0062] Specifically, calculating the overall loss function value includes:
[0063] S41. Based on the dominant hatred characteristics output by the lower-level group and speech encoding features f i Calculate the contrast loss function value of the low-level group;
[0064] In the low-level group (i.e., layers 1-4), the focus is on learning the difference between explicit hate speech and ordinary hate speech. Through low-level feature extraction, the model is able to capture the direct offensiveness of explicit hate speech. The contrast loss function L for this layer group is early as follows:
[0065]
[0066] Among them, f i n,min is a negative sample, that is, the hate speech encoding feature f i h The encoding features of normal speech with the minimum entropy, sim is the similarity calculation, and τ is the temperature parameter.
[0067] The entropy H(f i h ,f i n )=-∑p(f i h )logq(f i n ); where p(f i h ) represents the hate speech encoding feature f i h The probability distribution of q(f i n ) represents the normal speech encoding feature f i n The probability distribution of .
[0068] S42. Based on the dominant hatred characteristics output by the middle group and veiled hate speech and speech coding features f i Calculate the contrast loss function value of the middle group;
[0069] Unfreeze the weights of the middle layer group (layers 5-8) to learn more complex implicit hate speech. The model can understand complex semantic features such as irony and metaphor. The contrast loss function L mid as follows:
[0070]
[0071] S43. Based on the dominant hatred characteristics output by the high-level group implicit hate speech and characteristics of confrontational hate speech and speech coding features f i Calculate the contrast loss function value of the high-level group;
[0072] Unfreeze the high-level group (layers 9-12), and introduce the adversarial samples generated by the multi-level generation module into the model for contrastive learning to enhance the model's robustness against camouflage attacks. The contrast loss function L for this layer group is final as follows:
[0073]
[0074] S34. Combine the low-layer contrast loss function value, the middle-layer contrast loss function value, and the high-layer contrast loss function value to obtain the total loss function value.
[0075] The total loss function is as follows:
[0076] L total =α×L early +β×L mid +γ×L final
[0077] Among them, L total is the total contrast loss, α, β, and γ are weight coefficients at different stages, satisfying α+β+γ=1.
[0078] In one embodiment, the speech set is divided into a training set and a validation set. The implicit hate speech recognition model is trained based on the training set, and the F1 score is calculated on the validation set to evaluate the model's performance. If the F1 score meets expectations, the final trained implicit hate speech recognition model is obtained; otherwise, the model architecture or hyperparameters are adjusted and the training process is repeated.
[0079] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation plans of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A hate recognition method combining multi-level hate generation and progressive contrastive learning, characterized by: include: Obtain speech, input the speech into the trained implicit hate recognition model, obtain speech features, and obtain hate recognition results based on the speech features; The implicit hatred recognition model includes: feature encoding module, multi-level hatred generation module and feature extraction module; The training process of the implicit hate detection model includes: S1. Obtain speech set D, which includes normal speech and hate speech; i Input feature encoding module to obtain speech encoding feature f i ; Speech coding features include: hate speech coding features f i h and normal speech encoding features f i n ; Where i represents the index of the speech; S2. Encode hate speech features f i h Input the multi-level hate generation module to get multi-level hate speech S3. Multi-layered hate speech Input the feature extraction module to obtain the multi-level hate speech feature F i h ; S4. Based on the multi-level hate speech characteristics F i h and speech encoding features f i Calculate the total loss function value, and update the parameters of the implicit hatred recognition model according to the total loss function value. When the loss function value is minimized, the trained implicit hatred recognition model is obtained.
2. The method for identifying hatred by combining multi-level hatred generation and progressive contrastive learning according to claim 1, characterized in that: The multi-level hate generation module includes: explicit generation module, implicit generation module and adversarial generation module; the multi-level hate generation module encodes the hate speech features f i h Processing includes: encoding hate speech features f i h Input the explicit generation module, implicit generation module and adversarial generation module respectively to obtain multi-level hate speech Among them, multi-level hate speech Including explicit hate speech implicit hate speech and confrontational hate speech 3. The method for identifying hatred by combining multi-level hatred generation and progressive contrastive learning according to claim 2, characterized in that: The explicit generation module encodes the features of hate speech f i h Processing includes: According to the hate speech coding feature f i h Generate multiple explicit hate generation samples using a pre-trained large language model; Obtaining a hate word library, and counting the hate words in each explicit hate generation sample based on the hate word library; performing weighted summation on the hate words in each explicit hate generation sample to obtain the offensive intensity of each explicit hate generation sample; Calculate each explicit hate generation sample and hate speech encoding feature f separately i h The corresponding original speech x i The semantic consistency of each explicit hatred generation sample is obtained; The offensive intensity and semantic consistency score of each explicit hate generation sample are integrated to obtain the total score of each explicit hate generation sample; Select the explicit hate speech sample with the highest total score as the optimal explicit hate speech 4. The method for identifying hatred by combining multi-level hatred generation and progressive contrastive learning according to claim 2, characterized in that: The implicit generation module encodes the features of hate speech f i h Processing includes: According to the hate speech coding feature f i h Use a pre-trained large language model to generate multiple implicit hate generation samples; Obtain a hate word library, count the non-hate words in each implicit hate generation sample based on the hate word library, calculate the proportion of non-hate words in each implicit hate generation sample, and obtain a metaphorical score for each implicit hate generation sample; Calculate the potential offensiveness score of each implicit hate generation sample; Calculate each implicit hatred generation sample and hate speech encoding feature f separately i h The corresponding original speech x i The semantic consistency of each implicit hatred generation sample is obtained; The metaphorical score, potential offensive score, and semantic consistency score of each implicit hate generation sample are integrated to obtain the total score of the implicit hate generation sample; The implicit hate speech generated sample with the highest total score is regarded as the optimal implicit hate speech.
5. The method for identifying hatred by combining multi-level hatred generation and progressive contrastive learning according to claim 2, characterized in that: Adversarial generation module encodes features f of hate speech i h Processing includes: According to the hate speech coding feature f i h Generate multiple adversarial hate generation samples using a pre-trained large language model; Calculate the adversarial score of each adversarially generated hate sample using the hate detection model; Calculate the metaphoricality score of each adversarial hate generation sample; Calculate each adversarial hatred generation sample and hate speech encoding feature f separately i h The corresponding original speech x i The semantic consistency of each adversarial hatred generation sample is obtained; The adversarial score, metaphor score, and semantic consistency score of each adversarial hate generation sample are integrated to obtain the total score of each adversarial hate generation sample; The adversarial hate speech generated sample with the highest total score is regarded as the optimal adversarial hate speech.
6. The method for identifying hatred by combining multi-level hatred generation and progressive contrastive learning according to claim 1, characterized in that: The feature extraction module is the Transform network; the Transform network includes twelve network layers; among them, the 1st to 4th network layers are the bottom group, the 5th to 8th network layers are the middle group, and the 9th to 12th network layers are the high group.
7. The method for identifying hatred by combining multi-level hatred generation and progressive contrastive learning according to claim 1, characterized in that: Multi-layered characteristics of hate speech in, Represents the multi-level hate speech features output by the bottom group, middle group, and high-level group of the feature extraction module; the multi-level hate speech features of each group c include: explicit hate speech features Characteristics of implicit hate speech and characteristics of confrontational hate speech Calculating the overall loss function value includes: S41. Based on the dominant hatred characteristics output by the low-level group and speech encoding features f i Calculate the contrast loss function value of the low-level group; S42. Based on the dominant hatred characteristics output by the middle group and veiled hate speech and speech coding features f i Calculate the contrast loss function value of the middle group; S43. Based on the dominant hatred characteristics output by the high-level group implicit hate speech and characteristics of confrontational hate speech and speech coding features f i Calculate the contrast loss function value of the high-level group; S44. Combine the low-level group contrast loss function value, the middle-level group contrast loss function value, and the high-level group contrast loss function value to obtain a total loss function value.
8. The method for identifying hatred by combining multi-level hatred generation and progressive contrastive learning according to claim 7, characterized in that: Low-level group contrast loss function K early for: Among them, f i n,min is a negative sample, that is, the hate speech encoding feature f i h The encoding features of normal speech with the minimum entropy, sim is the similarity calculation, and τ is the temperature parameter.
9. The method for identifying hatred by combining multi-level hatred generation and progressive contrastive learning according to claim 7, characterized in that: The contrast loss function value L of the middle group mid for: Among them, f i n,min is a negative sample, that is, the hate speech encoding feature f i h The encoding features of normal speech with the minimum entropy, sim is the similarity calculation, and τ is the temperature parameter.
10. The method for identifying hatred by combining multi-level hatred generation and progressive contrastive learning according to claim 7, characterized in that: The contrast loss function value L of the high-level group final for: Among them, f i n,min is a negative sample, that is, the hate speech encoding feature f i h The encoding features of normal speech with the minimum entropy, sim is the similarity calculation, and τ is the temperature parameter.