A method and system for detecting hate content in videos based on multi-agent collaboration

Abstract

This invention discloses a method and system for detecting hate content in videos based on multi-agent collaboration. The detection method includes: S1 Data preprocessing: performing multimodal data processing on the input video content; S2 Opposite perspective agent debate: setting up supporter and opponent agents, both of which are used to capture semantic cues in the video content and generate explanatory evidence; S3 Evidence synthesis: setting up a synthesis agent to integrate the explanatory evidence and generate a comprehensive explanation; S4 Reason enhancement prediction: fusing multimodal features and reason features, and performing prediction through a two-layer multilayer perceptron; S5 Hate score calculation: calculating a hate score based on the predicted content, and determining whether the video content contains hate content by comparing it with a preset threshold. This invention, through multi-agent collaboration and reason enhancement prediction mechanism, can effectively identify subtle hate content in video content, while providing a credible and comprehensive explanation.

Description

Technical Field

[0001] This invention belongs to the field of video content analysis technology, and relates to a method and system for detecting hate content in videos based on multi-agent collaboration. Background Technology

[0002] With the rise of short video platforms (such as TikTok and Instagram Reels), online videos have become a primary way for people to obtain information and entertainment in their daily lives. Due to their strong appeal and emotional impact, videos have also become an efficient medium for spreading hate content. Therefore, developing automated video hate content detection systems has become crucial for maintaining a healthy online environment.

[0003] Early methods for video hate detection primarily relied on traditional machine learning models, such as Support Vector Machines (SVMs) and Random Forests, to process multimodal data. Recent research has shifted towards leveraging pre-trained multimodal models, such as BERT, ViT, and HuBERT, combining modality fusion components and task-specific classifiers to enhance detection performance. While these methods significantly improve detection performance by integrating visual, textual, and audio features, they generally suffer from a significant limitation: they rely on black-box models and cannot provide explanations for classifying videos as hate content. This deficiency severely weakens the reliability and applicability of these methods on video-sharing platforms.

[0004] Current research on multimodal hate detection primarily focuses on hate memes, largely thanks to extensive benchmark datasets. These studies typically use pre-trained language and visual encoders to extract features, then employ multimodal fusion strategies for detection. Video hate detection, however, has historically relied on classic machine learning techniques and deep learning architectures such as LSTM and RNN. The introduction of high-quality benchmark datasets (such as HateMM and MultiHateClip) has propelled the development of video hate detection, providing data and baselines for integrating multimodal features for prediction.

[0005] Recent attempts to provide explanations for hate detection have primarily focused on the domain of hate memes. For example, ExplainHM utilizes debate between two large language models (LLMs) to generate explanations for harmful memes. However, these methods often neglect well-structured thought chain reasoning and effective communication between LLMs, limiting their application beyond relatively simple hate memes. When applied to complex video hate detection, these methods tend to result in shallow understandings and incoherent explanations.

[0006] In summary, current video hate detection methods suffer from the following main problems: 1. Lack of deep understanding of subtle and complex hate elements in videos; 2. Inability to provide clear and credible explanations to support their detection decisions; 3. Limited generalization ability when handling cross-platform and diverse hate content; 4. Inability to effectively integrate multiple interpretive perspectives to provide a comprehensive content assessment. To address these issues, there is an urgent need to propose a detection method and system for video hate content that is interpretable based on multi-agent collaboration. Summary of the Invention

[0007] To address the aforementioned problems, this invention proposes a method and system for detecting interpretable hate content in videos based on multi-agent collaboration, which effectively solves the problems in the prior art.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows: a method for detecting interpretable hate content in videos based on multi-agent cooperation, comprising the following steps:

[0009] S1. Data preprocessing: Perform multimodal data processing on the input video content, extract audio, visual and text modal information, and then convert them into text information and concatenate them into a complete text representation;

[0010] S2, Opposite Perspective Agent Debate: Set up supporter agents and opponent agents, both of which are used to capture semantic cues in video content and generate interpretive evidence; supporter agents argue for hate classification in video content, while opponent agents argue for benign classification in video content;

[0011] S3. Evidence Synthesis: Set up a synthesis agent to integrate the explanatory evidence generated by agents with opposite perspectives in step S2, extract credible evidence, and finally generate a comprehensive explanation.

[0012] S4. Reason Addition Prediction: Extract reason features from the comprehensive explanation in step S3 using a pre-trained model, and extract multimodal features from the video content in step S1. Use a feedforward neural network of a specific modality to unify the feature dimensions, fuse multimodal features and reason features, and finally make predictions through a two-layer multilayer perceptron.

[0013] S5. Hate Score Calculation: Calculate the hate score based on the predicted content output in step S4, and determine whether the video content contains hate content by comparing it with a preset threshold.

[0014] Furthermore, both the supporter and opponent agents first analyze visual content during the assertion process, and then use the complete text representation for cross-modal verification; by setting up a two-step thought chain reasoning process, subtle clues supporting hate classification and benign classification are fully unearthed.

[0015] Furthermore, the assertion process of the supporting agent includes the following steps:

[0016] S2.1, Supporter agent analyzes visual content Identify subtle elements that support hate classification and provide the identification results. It is expressed by the following formula:

[0017]

[0018] Prompt 1 states: "As a proponent of the claim that videos contain hate signals, I examine visual content..." Your task is to identify and highlight any subtle elements that support this claim. for;

[0019] S2.2, The supporter agent will use the identification results from step S2.1. Cross-validation with the complete text representation generates explanatory evidence. It is expressed by the following formula:

[0020]

[0021] Prompt2 states: "As a proponent of the claim that videos contain hate signals, based on visual content..." The obtained recognition results Examine the full text representation Your task is to verify the recognition results. With full text representation And identify additional nuances that support the claim, providing comprehensive and persuasive explanatory evidence;

[0022] The opposing agent follows the above process to identify explanatory evidence supporting benign classification from the opposite perspective.

[0023] Furthermore, step S3 also includes the following steps:

[0024] S3.1, The explanatory evidence generated in step S2 and explanatory evidence Integrate and generate a comprehensive explanation It is expressed by the following formula:

[0025]

[0026] Prompt 3 states: "As a synthesizer, examine interpretive evidence." and explanatory evidence Your task is to analyze the above explanatory evidence, highlight the key evidence, discard insufficient claims, and provide a comprehensive and convincing final explanation for the hate video classification.

[0027] Furthermore, the pre-trained model in step S4 is a pre-trained BERT model.

[0028] Furthermore, the multimodal features in step S4 include audio features, text features, and visual features.

[0029] This invention also proposes a multi-intelligence collaborative interpretable video hate content detection system, comprising:

[0030] The data preprocessing module is used to perform multimodal data processing on video content and generate a complete text representation;

[0031] A multi-agent collaboration module is used to capture semantic cues in video content from opposite perspectives and generate interpretive evidence;

[0032] The evidence integration module is used to integrate the generated interpretive evidence into a comprehensive interpretation;

[0033] The reason enhancement prediction module is used to extract reason features from comprehensive explanations and multimodal features from video content, and then fuse them to output predicted content.

[0034] The determination module is used to calculate the hate score of the output predicted content and compare it with a preset threshold to determine whether it contains hate content.

[0035] Furthermore, supporter agents are used to identify and extract interpretive evidence from video content that supports hate classification;

[0036] The opponent agent is used to identify and extract explanatory evidence from video content that supports benign classification;

[0037] The synthesizer agent is used to integrate the explanatory evidence from the supporter and opponent agents to generate a comprehensive explanation.

[0038] Furthermore, the feature extraction submodule is used to extract rationale features from the comprehensive explanation and to extract multimodal features from the video content;

[0039] The feature fusion submodule is used to unify feature dimensions through a feedforward neural network of a specific modality, and to fuse multimodal features and rational features;

[0040] The prediction submodule is used to perform final classification on the fused features and output the predicted content.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. In this invention, through the framework of three roles of intelligent agents and the reason-enhanced prediction mechanism, it is possible to effectively identify veiled hate content in video content, while providing a credible and comprehensive explanation, and improving the accuracy, transparency and acceptability of hate detection in video content;

[0043] 2. In this invention, within the framework of a large-scale multimodal model, three types of intelligent agents and an evidence synthesis mechanism are introduced to achieve a deep understanding and comprehensive interpretation of hate content in video content, thus avoiding the problem of one-sided interpretation caused by a single perspective.

[0044] 3. In this invention, the reasoning process of three types of intelligent agents enables in-depth analysis and interpretation of video content, ensuring that the model's decision-making process is highly transparent and interpretable. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the overall framework of the present invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] like Figure 1 As shown, this invention proposes a method for detecting interpretable hate content in videos based on multi-agent cooperation, comprising the following steps:

[0048] S1. Data Preprocessing: Multimodal data processing is performed on the input video content to extract audio, visual, and text modal information, which are then converted into text information and concatenated into a complete text representation. Specifically, for the audio modality, the audio track is extracted; for the visual modality, M frames are sampled uniformly; for the text modality, in addition to obtaining the video title, Whisper is used to generate transcribed text from the audio, and PaddleOCR is used to extract screen text from the sampled frames.

[0049] S2, Opposite Perspective Agent Debate: Set up supporter agents and opponent agents, both of which are used to capture semantic cues in video content and generate interpretive evidence; supporter agents argue for hate classification in video content, while opponent agents argue for benign classification in video content;

[0050] In this embodiment, both the supporter and the opponent agents first analyze the visual content during the assertion process, and then use the complete text representation for cross-modal verification; by setting a two-step thought chain reasoning process, the subtle clues supporting hate classification and support benign classification are fully discovered;

[0051] Specifically, the supporter agent, opponent agent, and integrated agent of this invention all employ a large-scale multimodal model (LMM) as the system foundation, enabling each agent to understand and analyze the visual and textual content in the video; for example, using the Doubao large model, Deepseek large model, ChatGPT, etc.; and the following formula appears All of them represent large-scale multimodal models.

[0052] The assertion process of the supporting agent includes the following steps:

[0053] S2.1, Supporter agent analyzes visual content Identify subtle elements that support hate classification and provide the identification results. It is expressed by the following formula:

[0054]

[0055] Prompt 1 states: "As a proponent of the claim that videos contain hate signals, I examine visual content..." Your task is to identify and highlight any subtle elements that support this claim.

[0056] S2.2, The supporter agent will use the identification results from step S2.1. Cross-validation with the complete text representation generates explanatory evidence. It is expressed by the following formula:

[0057]

[0058] Prompt2 states: "As a proponent of the claim that videos contain hate signals, based on visual content..." The obtained recognition results Examine the full text representation Your task is to verify the recognition results. With full text representation And identify additional nuances that support the claim, providing comprehensive and persuasive explanatory evidence;

[0059] The opposing agent follows the above process to identify explanatory evidence supporting benign classification from the opposite perspective.

[0060] S3. Evidence Synthesis: A synthesis agent is set up to integrate the explanatory evidence generated by agents with opposing perspectives in step S2, refine credible evidence, and ultimately generate a comprehensive explanation; that is, to execute step S3.1:

[0061] S3.1, The explanatory evidence generated in step S2 and explanatory evidence Integrate and generate a comprehensive explanation It is expressed by the following formula:

[0062]

[0063] Prompt 3 states: "As a synthesizer, examine interpretive evidence." and explanatory evidence Your task is to analyze the above explanatory evidence, highlight the key evidence, discard insufficient claims, and provide a comprehensive and convincing final explanation for the hate video classification.

[0064] S4. Reason Addition Prediction: Extract reason features from the comprehensive explanation in step S3 using a pre-trained model, and extract multimodal features from the video content in step S1. Use a feedforward neural network of a specific modality to unify the feature dimensions, fuse multimodal features and reason features, and finally make predictions through a two-layer multilayer perceptron. The feedforward neural network of the specific modality is selected according to the different modalities in the reason features and multimodal features.

[0065] Specifically, the rational features are first obtained from step S3 using the pre-trained BERT model encoding, represented as... Among them Ψ t It is the BERT model;

[0066] Simultaneously, multimodal features are extracted from the original video content: audio features are obtained by calculating Mel-frequency cepstral coefficients (MFCCs), denoted as... Text features are extracted by the BERT model and are represented as follows: Visual features are extracted by ViT and represented as The aforementioned multimodal features and rational features are processed through feedforward neural networks (FFNs) of a specific modality, and their dimension is unified to d, represented as follows: Where m∈{r,a,t,v};

[0067] Next, the predicted content is output. The prediction process is as follows:

[0068]

[0069] The Predictor is implemented using two layers of Multilayer Perceptron (MLP). This indicates a connection operation. It's worth noting that during the training of the above model, the pre-trained modal encoder Ψ... a Ψ v and Ψ t The parameters are kept frozen, and optimization is performed using binary cross-entropy loss.

[0070] S5. Hate Score Calculation: Calculate the hate score based on the predicted content output in step S4, and determine whether the video content contains hate content by comparing it with a preset threshold; specifically, if the hate score exceeds the preset threshold, it can be considered to contain hate content.

[0071] In this invention, through a framework of three intelligent agents and a reason-enhanced prediction mechanism, it is possible to effectively identify veiled hate content in video content, while providing a credible and comprehensive explanation, thus enhancing the accuracy, transparency, and acceptability of hate detection in video content.

[0072] This invention also proposes a multi-agent collaborative system for detecting interpretable hate content in videos, comprising:

[0073] The data preprocessing module is used to perform multimodal data processing on video content and generate a complete text representation;

[0074] A multi-agent collaboration module is used to capture semantic cues in video content from opposite perspectives and generate interpretive evidence;

[0075] The evidence integration module is used to integrate the generated interpretive evidence into a comprehensive interpretation;

[0076] The reason enhancement prediction module is used to extract reason features from comprehensive explanations and multimodal features from video content, and then fuse them to output predicted content.

[0077] The determination module is used to calculate the hate score of the output predicted content and compare it with a preset threshold to determine whether it contains hate content.

[0078] Furthermore, supporter agents are used to identify and extract interpretive evidence from video content that supports hate classification;

[0079] The opponent agent is used to identify and extract explanatory evidence from video content that supports benign classification;

[0080] The synthesizer agent is used to integrate the explanatory evidence from the supporter and opponent agents to generate a comprehensive explanation.

[0081] Furthermore, the feature extraction submodule is used to extract rationale features from the comprehensive explanation and to extract multimodal features from the video content;

[0082] The feature fusion submodule is used to unify feature dimensions through a feedforward neural network of a specific modality, and to fuse multimodal features and rational features;

[0083] The prediction submodule is used to perform final classification on the fused features and output the predicted content.

[0084] The invention was also experimentally verified to assess its overall performance. The verification process is as follows:

[0085] 1. Experimental setup

[0086] a) Dataset

[0087] To verify the experimental effects of the invention on multiple different types of datasets, two real-world video datasets were used for comprehensive evaluation, including MultiHateClip YouTube (MHClipEN) and MultiHateClip Bilibili (MHClipZH).

[0088] The MHClipEN and MHClipZH datasets are specifically designed to detect hate videos on YouTube and Bilibili (both popular video platforms). Each entry contains the video, title, transcribed text, and detailed annotations. These annotations provide comprehensive information, such as whether the video is categorized as hateful, offensive, or non-hateful. Hateful and offensive content are both classified as malicious using a binary classification method.

[0089] b) Measurement Standards

[0090] To comprehensively evaluate the performance of this invention, four metrics were used: classification accuracy (ACC), macro F1 score (M-F1), macro precision (MP), and macro recall (MR). These metrics can comprehensively reflect the model's performance in the hate video detection task.

[0091] c) Baseline Model

[0092] To evaluate the performance of this invention (MATCH), it was compared with six state-of-the-art models, which can be divided into two groups:

[0093] ①. Single-modal hate detection methods: Methods that use only single-modal information for detection, such as BERT, ViT, HuBERT and TSformer.

[0094] ②. Multimodal hate detection methods: Methods that integrate multimodal information for detection, such as HTMM and MHCL.

[0095] 2. Experimental Results

[0096] Under the same experimental settings, the experimental results of this invention (MATCH) were compared with those of other benchmark models. The figure below shows the comparison of different models on the same metrics ACC, M-F1, MP, and MR across three datasets, with the best results in each column highlighted in bold and the second-best results underlined:

[0097]

[0098] Table 1: Experimental results of different models on two datasets

[0099] The results show that the present invention (MATCH) outperforms all competing baselines on all three datasets, demonstrating its superiority in effectively detecting implicit hate content in videos. The insightful evidence generated through multi-agent collaboration provides valuable supplementary knowledge, enabling the model to detect deeper hate patterns and significantly improving prediction accuracy.

[0100] Single-modal detection methods performed the worst, demonstrating their inability to handle hate videos containing complex modal features. Multimodal detection methods significantly outperformed single-modal methods, highlighting the importance of utilizing all modalities in video hate detection.

[0101] However, the reliance on direct prediction by baseline methods limits their adaptability to task-specific challenges and often fails to capture subtle hate elements in videos. In contrast, this invention (MATCH) introduces a multi-agent collaborative framework combined with reason-enhanced prediction, achieving a robust integration of subtle reasoning and cross-modal evidence for more accurate and context-aware hate element detection.

[0102] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting explainable video hate content based on multi-agent collaboration, characterized in that, The method comprises the following steps: S1, data preprocessing: multi-modal data processing is performed on the input video content, modal information of audio, vision and text is extracted, and then all are converted into text information and connected into complete text representation; S2, opposite-view agent debate: a supporter agent and an opponent agent are set, both of which are used to capture semantic clues in the video content and generate explanatory evidence; the supporter agent advocates the hate classification in the video content, and the opponent agent advocates the benign classification in the video content; S3, evidence synthesis: a synthesis agent is set, which is used to integrate the explanatory evidence generated by the agents with opposite views in step S2, refine the reliable evidence, and finally generate a comprehensive explanation; S4, reason increase prediction: reason features are extracted from the comprehensive explanation in step S3 by using a pre-trained model, multi-modal features are extracted from the video content in step S1, the feature dimensions are unified by using a feedforward neural network of a specific mode, the multi-modal features and the reason features are fused, and finally two-layer perceptrons are used for prediction; S5, hate score calculation: the hate score is calculated according to the prediction content output in step S4, and whether the video content contains hate content is determined by comparing with a preset threshold.

2. The method of claim 1, wherein the method is based on multi-agent collaboration for explainable video hate content detection. In the process of advocating of the supporter agent and the opponent agent, the visual content is first analyzed, and then cross-modal verification is performed by using the complete text representation. 3.The method of claim 1, wherein, The process of advocating of the supporter agent comprises the following steps: S2.1, supporter agent analyzes visual content Identify subtle elements that support the hate classification, give the recognition result which is expressed by the following formula: where Prompt1 is: "As a supporter of the claim that a video contains hate speech, scrutinize the visual content for any subtle elements that support this claim"; for large multi-modal models; S2.

2. The supporter agent will identify the result in step S2.1 Cross-validated with the full text representation, generating explanatory evidence which is expressed by the formula: where Prompt2 is: "As a supporter who claims that a video contains hate signals, provide a comprehensive and persuasive explanatory evidence based on the visual content the resulting recognition examine the full text representation your task is to verify the recognition result with the full text representation and identify additional subtle elements that support the claim, providing comprehensive and persuasive explanatory evidence" The counter agent follows the above process to identify explanatory evidence supporting benign classification from the opposite perspective 4. The method of claim 3, wherein the method is based on multi-agent collaboration for explainable video hate content detection. The step S3 further comprises the following steps: S3.1, integrating the explanatory evidence generated in said step S2 and the explanatory evidence to generate a comprehensive explanation which is expressed by the formula: where Prompt3 is: "As a synthesizer, examine the interpretive evidence and interpretive evidence Your task is to analyze the above interpretive evidence, highlight the important evidence, discard the insufficient claims, and provide a comprehensive and persuasive final explanation for the hate video classification"; for large multimodal models.

5. The method of claim 1, wherein the method is based on multi-agent collaboration for explainable video hate content detection. The pre-trained model in the step S4 is a pre-trained BERT model.

6. The method of claim 1, wherein the method is based on multi-agent collaboration for explainable video hate content detection. The multi-modal features in the step S4 include audio features, text features and visual features.

7. A detection system characterized by, The method for detecting hate content in a video based on multi-agent cooperation according to any one of claims 1-6 comprises: a data preprocessing module, configured to perform multi-modal data processing on the video content and generate complete text representation; a multi-agent cooperation module, configured to capture semantic clues in the video content from opposite views and generate explanatory evidence; an evidence synthesis module, configured to integrate the generated explanatory evidence into a comprehensive explanation; a reason enhancement prediction module, configured to extract reason features from the comprehensive explanation and multi-modal features from the video content, and output prediction content after fusion; a determination module, configured to calculate the hate score of the output prediction content, and compare with a preset threshold to determine whether the video content contains hate content.

8. The detection system of claim 7, wherein, The multi-agent cooperation module comprises: a supporter agent, configured to identify and extract explanatory evidence supporting the hate classification from the video content; an opponent agent, configured to identify and extract explanatory evidence supporting the benign classification from the video content; a synthesizer agent, configured to integrate the explanatory evidence of the supporter agent and the opponent agent to generate a comprehensive explanation.

9. The detection system of claim 7, wherein, The reason enhancement prediction module comprises: a feature extraction submodule, configured to extract reason features from the comprehensive explanation and multi-modal features from the video content; a feature fusion submodule, configured to unify the feature dimensions by using a feedforward neural network of a specific mode, and fuse the multi-modal features and the reason features; a prediction submodule, configured to finally classify the fused features and output prediction content.

Images