Detection method and system capable of explaining video hatred content based on multi-agent cooperation

Abstract

The invention discloses a detection method and system capable of explaining video hatred content based on multi-agent cooperation. The detection method comprises the following steps: S1, data preprocessing: carrying out multi-modal data processing on input video content; s2, performing opposite-view-angle agent debate: setting a supporter agent and a counter agent, wherein the supporter agent and the counter agent are simultaneously used for capturing semantic clues in video contents and generating explanatory evidences; s3, evidence synthesis: setting a comprehensive agent to integrate the explanatory evidence, and generating comprehensive explanation; s4, reason increase prediction: fusing the multi-modal features and reason features, and performing prediction through a two-layer and multi-layer perceptron; s5, calculating a hatred score: calculating the hatred score according to the predicted content, and determining whether the video content contains hatred content according to a preset threshold value; according to the method, through multi-agent cooperation and a reason enhancement prediction mechanism, the private hatred content in the video content can be effectively identified, and meanwhile, credible and comprehensive explanation is provided.

Description

Technical Field

[0001] The present invention belongs to the technical field of video content analysis, and relates to a detection method and system for interpretable hateful content in videos based on multi-agent collaboration. Background Art

[0002] With the rise of short-form video platforms like TikTok and Instagram Reels, online video has become a primary way people access information and entertainment in their daily lives. Due to their strong appeal and emotional appeal, videos have also become a highly effective medium for spreading hateful content. Therefore, developing automated systems to detect hateful content in videos has become crucial for maintaining a healthy online environment.

[0003] Early approaches to video hate detection primarily relied on traditional machine learning models, such as support vector machines (SVMs) and random forests, to process multimodal data. Recent research has shifted toward leveraging pre-trained multimodal models, such as BERT, ViT, and HuBERT, combined with modal fusion components and task-specific classifiers to enhance detection performance. While these approaches significantly improve detection performance by integrating visual, textual, and audio features, they share a significant limitation: their reliance on black-box models and their inability to provide explanations for classifying videos as hateful. This drawback severely undermines the reliability and applicability of these approaches on video sharing platforms.

[0004] Current multimodal hate detection research primarily focuses on hate memes, largely benefiting from the availability of extensive meme benchmark datasets. These studies typically use pre-trained language and visual encoders to extract features, then employ multimodal fusion strategies for detection. Video hate detection, on the other hand, has historically relied on classic machine learning techniques and deep learning architectures such as LSTM and RNN. The introduction of high-quality benchmark datasets such as HateMM and MultiHateClip has propelled the development of video hate detection, providing data and baselines for integrating multimodal features for prediction.

[0005] Recent attempts to provide explanations for hate detection have primarily focused on the field of hate memes. For example, ExplainHM leverages a debate between two large language models (LLMs) to generate explanations for harmful memes. However, these methods often neglect well-structured thought chain reasoning and effective communication between LLMs, limiting their applicability beyond relatively simple hate memes. When applied to complex video hate detection, these methods often result in shallow understanding and incoherent explanations.

[0006] In summary, current methods for detecting hateful content in videos suffer from the following key challenges: 1. They lack a deep understanding of subtle and complex hateful elements in videos; 2. They are unable to provide clear and credible explanations to support their detection decisions; 3. They have limited generalization capabilities when dealing with cross-platform and diverse hateful content; and 4. They are unable to effectively integrate multiple interpretation perspectives to provide a comprehensive assessment of content. To address these challenges, a method and system for detecting hateful content in videos that can be explained through multi-agent collaboration is urgently needed. Summary of the Invention

[0007] In response to the above problems, the present invention proposes a method and system for detecting hateful content in explainable videos based on multi-agent collaboration, which effectively solves the problems in the prior art.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: a method for detecting hateful content in explainable videos based on multi-agent collaboration, comprising the following steps:

[0009] S1. Data preprocessing: Perform multimodal data processing on the input video content, extract audio, visual, and textual modal information, and then convert them into text information and connect them into a complete text representation;

[0010] S2, Opposing Perspective Agent Debate: Set up a supporter agent and an opponent agent, both of which are used to capture semantic clues in the video content and generate explanatory evidence; the supporter agent advocates the hate classification in the video content, and the opponent agent advocates the benign classification in the video content;

[0011] S3, evidence synthesis: Set up a comprehensive agent to integrate the explanatory evidence generated by the agents with opposite perspectives in step S2, refine credible evidence, and finally generate a comprehensive explanation;

[0012] S4, reason addition prediction: Extract reason features from the comprehensive explanation in step S3 through a pre-trained model, and extract multimodal features from the video content in step S1. Use a modality-specific feedforward neural network to unify feature dimensions, fuse multimodal features and reason features, and finally make predictions using a two-layer multilayer perceptron.

[0013] S5. Calculate the hate score: Calculate the hate score based on the predicted content output in step S4, and compare it with a preset threshold to determine whether the video content contains hate content.

[0014] Furthermore, both the supporter agent and the opponent agent first analyze the visual content during their advocacy process, and then use the complete text representation for cross-modal verification; by setting up a two-step thought chain reasoning process, they comprehensively explore the implicit clues that support the hate classification and the benign classification.

[0015] Furthermore, the advocacy process of the supporter agent includes the following steps:

[0016] S2.1. Supporter Agent Analyzes Visual Content Identify subtle elements that support hate classification and provide identification results It is expressed by the following formula:

[0017]

[0018] Prompt 1 reads: "As a supporter of the claim that the video contains hate signals, examine the visual content. Your task is to identify and highlight any subtle elements that support this claim”; for;

[0019] S2.2, the supporter agent will recognize the results in step S2.1 Cross-validate with full text representation to generate explanatory evidence It is expressed by the following formula:

[0020]

[0021] Prompt 2 reads: "As a supporter of the claim that the video contains hate signals, based on the visual content The recognition results obtained Review the full text representation Your task is to verify the recognition results With full text representation and identifies additional subtle elements that support the claim, providing comprehensive and persuasive explanatory evidence”;

[0022] The opponent agent follows the above process to identify explanatory evidence supporting the benign classification from the opposite perspective

[0023] Furthermore, step S3 further includes the following steps:

[0024] S3.1. The explanatory evidence generated in step S2 and explanatory evidence Integrate and generate comprehensive explanations It is expressed by the following formula:

[0025]

[0026] Prompt 3 is: "As a synthesizer, examine the explanatory evidence and explanatory evidence Your task is to analyze the explanatory evidence presented above, highlight important evidence, discard weak claims, and provide a comprehensive and convincing final explanation for the hate video classification."

[0027] Furthermore, the pre-trained model in step S4 is a pre-trained BERT model.

[0028] Furthermore, the multimodal features in step S4 include audio features, text features, and visual features.

[0029] The present invention also proposes a detection system for hateful content in explainable videos based on multi-intelligence collaboration, comprising:

[0030] Data preprocessing module, used to perform multimodal data processing on video content and generate complete text representation;

[0031] A multi-agent collaboration module to capture semantic clues in video content from opposing perspectives and generate explanatory evidence;

[0032] The evidence synthesis module is used to integrate the generated explanatory evidence into a comprehensive explanation;

[0033] The reason enhancement prediction module is used to extract reason features from comprehensive explanations and multimodal features from video content, and then fuse them to output the predicted content;

[0034] The determination module is used to calculate the hate score of the output predicted content and compare it with a preset threshold to determine whether it contains hate content.

[0035] Furthermore, the supporter agent is used to identify and extract explanatory evidence supporting the classification of hate from video content;

[0036] Adversarial agents, which identify and extract explanatory evidence supporting benign classifications from video content;

[0037] The integrator agent is used to integrate the explanatory evidence of the supporter agent and the opponent agent to generate a comprehensive explanation.

[0038] Furthermore, the feature extraction submodule is used to propose reason features from the comprehensive explanation and extract multimodal features from the video content;

[0039] The feature fusion submodule is used to unify feature dimensions through a modality-specific feedforward neural network and fuse multimodal features with reason features;

[0040] The prediction submodule is used to perform final classification on the fused features and output the prediction content.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. This invention, through a framework of three intelligent agents and a reason-enhanced prediction mechanism, can effectively identify implicit hateful content in video content while providing credible and comprehensive explanations, thereby improving the accuracy, transparency, and acceptability of hateful content detection in video content.

[0043] 2. This invention introduces three intelligent agents and an evidence synthesis mechanism within the framework of a large-scale multimodal model, achieving a deep understanding and comprehensive interpretation of hateful content in video content, avoiding the one-sided interpretation problem caused by a single perspective;

[0044] 3. In the present invention, through the thought chain reasoning process of the three role intelligent agents, in-depth analysis and interpretation generation of the video content are achieved, ensuring that the decision-making process of the model is highly transparent and explainable. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the overall framework of the present invention. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] like Figure 1 As shown, the present invention proposes a method for detecting hateful content in explainable videos based on multi-agent collaboration, comprising the following steps:

[0048] S1. Data preprocessing: Perform multimodal data processing on the input video content, extracting audio, visual, and textual modal information, which is then converted into textual information and concatenated into a complete text representation. For the audio modality, the audio track is extracted; for the visual modality, M frames are uniformly sampled; for the text modality, in addition to obtaining the video title, Whisper is used to generate transcripts from the audio, and PaddleOCR is used to extract screen text from the sampled frames.

[0049] S2, Opposing Perspective Agent Debate: Set up a supporter agent and an opponent agent, both of which are used to capture semantic clues in the video content and generate explanatory evidence; the supporter agent advocates the hate classification in the video content, and the opponent agent advocates the benign classification in the video content;

[0050] In this embodiment, both the supporter and opponent agents first analyze the visual content during their advocacy process, and then use the complete text representation for cross-modal verification. By setting up a two-step thought chain reasoning process, they fully explore the implicit clues that support the hate classification and the benign classification.

[0051] Specifically, the supporter agent, the opponent agent, and the integrated agent of the present invention all use a large multimodal model (LMM) as the system foundation, so that each agent can understand and analyze the visual and textual content in the video; for example, the Doubao large model, the Deepseek large model, the ChatGPT, etc. are used; and the following formula appears Both represent large multimodal models.

[0052] The advocacy process of the supporter agent includes the following steps:

[0053] S2.1. Supporter Agent Analyzes Visual Content Identify subtle elements that support hate classification and provide identification results It is expressed by the following formula:

[0054]

[0055] Prompt 1 reads: "As a supporter of the claim that the video contains hate signals, examine the visual content. Your task is to identify and highlight any subtle elements that support this claim”;

[0056] S2.2, the supporter agent will recognize the results in step S2.1 Cross-validate with full text representation to generate explanatory evidence It is expressed by the following formula:

[0057]

[0058] Prompt 2 reads: "As a supporter of the claim that the video contains hate signals, based on the visual content The recognition results obtained Review the full text representation Your task is to verify the recognition results With full text representation and identifies additional subtle elements that support the claim, providing comprehensive and persuasive explanatory evidence”;

[0059] The opponent agent follows the above process to identify explanatory evidence supporting the benign classification from the opposite perspective.

[0060] S3. Evidence synthesis: Set up a comprehensive agent to integrate the explanatory evidence generated by the agents with opposite perspectives in step S2, refine the credible evidence, and ultimately generate a comprehensive explanation; that is, execute step S3.1:

[0061] S3.1. The explanatory evidence generated in step S2 and explanatory evidence Integrate and generate comprehensive explanations It is expressed by the following formula:

[0062]

[0063] Prompt 3 is: "As a synthesizer, examine the explanatory evidence and explanatory evidence Your task is to analyze the explanatory evidence presented above, highlight important evidence, discard weak claims, and provide a comprehensive and convincing final explanation for the hate video classification."

[0064] S4. Reason addition prediction: The pre-trained model extracts reason features from the comprehensive explanation in step S3, and simultaneously extracts multimodal features from the video content in step S1. A modality-specific feedforward neural network is used to unify feature dimensions, integrating multimodal features with reason features. Finally, a two-layer multilayer perceptron is used for prediction. The modality-specific feedforward neural network is selected based on the difference between the reason features and the modality in the multimodal features.

[0065] Specifically, we first use the pre-trained BERT model to encode the reason features from step S3, which is expressed as where Ψ t It is the BERT model;

[0066] At the same time, multimodal features are extracted from the original video content: the audio features are obtained by calculating the Mel-frequency cepstral coefficients (MFCCs), which are expressed as The text features are extracted by the BERT model and expressed as Visual features are extracted by ViT and expressed as The above multimodal features and reason features are processed by modality-specific feedforward neural networks (FFNs) and their dimensions are unified to d, which can be expressed as where m∈{r,a,t,v};

[0067] Next, the predicted content is output. The prediction process is as follows:

[0068]

[0069] The Predictor is implemented by two layers of multi-layer perceptron (MLP). Represents the connection operation. It is worth noting that when the above model is trained, the pre-trained modal encoder Ψ a ,Ψ v and Ψ t The parameters of are kept frozen and optimized using binary cross entropy loss.

[0070] S5. Calculate a hate score: Calculate a hate score based on the predicted content output in step S4, and compare it with a preset threshold to determine whether the video content contains hate content. Specifically, if the hate score exceeds the preset threshold, it is considered to contain hate content.

[0071] In the present invention, through the framework of three role agents and the prediction mechanism based on reason enhancement, it is possible to effectively identify implicit hateful content in video content, while providing credible and comprehensive explanations, and improving the accuracy, transparency and acceptability of hate detection in video content.

[0072] The present invention also proposes a detection system for hateful content in explainable videos based on multi-agent collaboration, comprising:

[0073] Data preprocessing module, used to perform multimodal data processing on video content and generate complete text representation;

[0074] A multi-agent collaboration module to capture semantic clues in video content from opposing perspectives and generate explanatory evidence;

[0075] The evidence synthesis module is used to integrate the generated explanatory evidence into a comprehensive explanation;

[0076] The reason enhancement prediction module is used to extract reason features from comprehensive explanations and multimodal features from video content, and then fuse them to output the predicted content;

[0077] The determination module is used to calculate the hate score of the output predicted content and compare it with a preset threshold to determine whether it contains hate content.

[0078] Furthermore, the supporter agent is used to identify and extract explanatory evidence supporting the classification of hate from video content;

[0079] Adversarial agents, which identify and extract explanatory evidence supporting benign classifications from video content;

[0080] The integrator agent is used to integrate the explanatory evidence of the supporter agent and the opponent agent to generate a comprehensive explanation.

[0081] Furthermore, the feature extraction submodule is used to propose reason features from the comprehensive explanation and extract multimodal features from the video content;

[0082] The feature fusion submodule is used to unify feature dimensions through a modality-specific feedforward neural network and fuse multimodal features with reason features;

[0083] The prediction submodule is used to perform final classification on the fused features and output the prediction content.

[0084] The present invention also conducted experimental verification to verify the comprehensive evaluation performance of the present invention, and the verification process is as follows:

[0085] 1. Experimental Setup

[0086] a) Dataset

[0087] In order to verify the experimental effect of the invention on multiple different types of datasets, two real-world video datasets were used for comprehensive evaluation, including MultiHateClip YouTube (MHClipEN) and MultiHateClip Bilibili (MHClipZH).

[0088] The MHClipEN and MHClipZH datasets are specifically designed for detecting hate videos on YouTube and Bilibili, two of the most popular video platforms. Each entry contains a video, title, transcript, and detailed annotations. These annotations provide comprehensive information, such as whether a video is classified as hateful, offensive, or non-hateful. Both hateful and offensive content are classified as malicious, using a binary classification approach.

[0089] b) Metrics

[0090] To comprehensively evaluate the performance of our method, we used four metrics: Classification Accuracy (ACC), Macro F1 Score (M-F1), Macro Precision (MP), and Macro Recall (MR). These metrics can fully reflect the performance of the model in the hate video detection task.

[0091] c) Baseline Model

[0092] To evaluate the performance of our invention (MATCH), we compare it with six state-of-the-art models, which can be divided into two groups:

[0093] ①. Single-modal hate detection method: a method that only uses a single modal information for detection, such as BERT, ViT, HuBERT, and TSformer.

[0094] ②. Multimodal hate detection methods: methods that integrate multimodal information for detection, such as HTMM and MHCL.

[0095] 2. Experimental Results

[0096] Under the same experimental settings, we compared the experimental results of our invention (MATCH) with other benchmark models. The following figure shows the comparison of the same metrics ACC, M-F1, MP, and MR across three datasets. The best result in each column is highlighted in bold, and the second-best result is underlined:

[0097]

[0098] Table 1: Experimental results of different models on two datasets

[0099] Results show that our proposed method (MATCH) outperforms all competing baselines on all three datasets, demonstrating its superiority in effectively detecting implicit hateful content in videos. The insights generated through multi-agent collaboration provide valuable complementary knowledge, enabling the model to detect deeper hateful patterns and significantly improve prediction accuracy.

[0100] Unimodal detection methods perform the worst, demonstrating their inability to handle hateful videos containing complex modal features. Multimodal detection methods significantly outperform unimodal methods, highlighting the importance of leveraging all modalities in video hate detection.

[0101] However, the reliance of baseline methods on direct prediction limits their adaptability to task-specific challenges and often fails to capture subtle hateful elements in videos. In contrast, our invention (MATCH) introduces a multi-agent collaborative framework combined with reason-enhanced prediction, enabling robust integration of subtle reasoning and cross-modal evidence for more accurate and context-aware hateful element detection.

[0102] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting hateful content in explainable videos based on multi-agent collaboration, characterized in that: The following steps are involved: S1. Data preprocessing: Perform multimodal data processing on the input video content, extract audio, visual, and textual modal information, and then convert them into text information and connect them into a complete text representation; S2, Opposing Perspective Agent Debate: Set up a supporter agent and an opponent agent, both of which are used to capture semantic clues in the video content and generate explanatory evidence; the supporter agent advocates the hate classification in the video content, and the opponent agent advocates the benign classification in the video content; S3, evidence synthesis: Set up a comprehensive agent to integrate the explanatory evidence generated by the agents with opposite perspectives in step S2, refine credible evidence, and finally generate a comprehensive explanation; S4, reason addition prediction: Extract reason features from the comprehensive explanation in step S3 through a pre-trained model, and extract multimodal features from the video content in step S1. Use a modality-specific feedforward neural network to unify feature dimensions, fuse multimodal features and reason features, and finally make predictions using a two-layer multilayer perceptron. S5. Calculate the hate score: Calculate the hate score based on the predicted content output in step S4, and compare it with a preset threshold to determine whether the video content contains hate content.

2. The method for detecting hateful content in explainable videos based on multi-agent collaboration according to claim 1, characterized in that: Both the supporter and opponent agents first analyze the visual content during their advocacy process and then perform cross-modal verification using the full text representation.

3. The method for detecting hateful content in explainable videos based on multi-agent collaboration according to claim 1, characterized in that: The advocacy process of the supporter agent includes the following steps: S2.

1. Supporter Agent Analyzes Visual Content Identify subtle elements that support hate classification and provide identification results It is expressed by the following formula: Prompt 1 reads: "As a supporter of the claim that the video contains hate signals, examine the visual content. Your task is to identify and highlight any subtle elements that support this claim”; For large multimodal models; S2.2, the supporter agent will recognize the results in step S2.1 Cross-validate with full text representation to generate explanatory evidence It is expressed by the following formula: Prompt 2 is: "As a supporter of the claim that the video contains hate signals, based on the visual content The recognition results obtained Review the full text representation Your task is to verify the recognition results With full text representation and identifies additional subtle elements that support the claim, providing comprehensive and persuasive explanatory evidence”; The opponent agent follows the above process to identify explanatory evidence supporting the benign classification from the opposite perspective 4. The method for detecting hateful content in explainable videos based on multi-agent collaboration according to claim 3, characterized in that: The step S3 further includes the following steps: S3.

1. The explanatory evidence generated in step S2 and explanatory evidence Integrate and generate comprehensive explanations It is expressed by the following formula: Prompt 3 is: "As a synthesizer, examine the explanatory evidence and explanatory evidence Your task is to analyze the explanatory evidence presented above, highlight the important evidence, discard the weak claims, and provide a comprehensive and convincing final explanation for the hate video classification”; For large multimodal models.

5. The method for detecting hateful content in explainable videos based on multi-agent collaboration according to claim 1, characterized in that: The pre-trained model in step S4 is a pre-trained BERT model.

6. The method for detecting hateful content in explainable videos based on multi-agent collaboration according to claim 1, characterized in that: The multimodal features in step S4 include audio features, text features, and visual features.

7. A detection system, characterized in that: The method for detecting hateful content in explainable videos based on multi-agent collaboration according to any one of claims 1 to 6 comprises: Data preprocessing module, used to perform multimodal data processing on video content and generate complete text representation; A multi-agent collaboration module to capture semantic clues in video content from opposing perspectives and generate explanatory evidence; The evidence synthesis module is used to integrate the generated explanatory evidence into a comprehensive explanation; The reason enhancement prediction module is used to extract reason features from comprehensive explanations and multimodal features from video content, and then fuse them to output the predicted content; The determination module is used to calculate the hate score of the output predicted content and compare it with a preset threshold to determine whether it contains hate content.

8. The detection system according to claim 7, characterized in that: The multi-agent collaboration module includes: Supporter Agent, which is used to identify and extract explanatory evidence supporting hate classification from video content; Adversarial agents, which identify and extract explanatory evidence supporting benign classifications from video content; The integrator agent is used to integrate the explanatory evidence of the supporter agent and the opponent agent to generate a comprehensive explanation.

9. The detection system according to claim 7, characterized in that: The reason enhancement prediction module includes: Feature extraction submodule, which is used to propose reason features from comprehensive explanations and extract multimodal features from video content; The feature fusion submodule is used to unify feature dimensions through a modality-specific feedforward neural network and fuse multimodal features with reason features; The prediction submodule is used to perform final classification on the fused features and output the prediction content.

Images