Chinese network violent event data set construction method based on man-machine cooperation

Abstract

The invention discloses a Chinese network violent event data set construction method based on man-machine collaboration, which comprises the following steps: 1) extracting comment data related to network violent events from a plurality of Chinese social media platforms, and ensuring data diversity and complexity; 2) adopting three network violence detection methods based on paraphrasing, thinking chain and multiple agents to generate pseudo labels and corresponding interpretation contents, and combining results of the three detection methods through an integration method; 3) performing manual labeling by a plurality of Chinese native language persons according to the generated pseudo labels and explanations to ensure the labeling accuracy; and 4) constructing an event-based Chinese network violent detection data set CHNCI according to a labeling result, and carrying out statistics on basic information of the data set. According to the method, the Chinese network violent detection data set based on the events is constructed in a mode of combining machine generation of pseudo labels and manual annotation, the data annotation cost is remarkably reduced, and meanwhile the coverage range and quality of the data set are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and particularly to a method for constructing a Chinese cyberbullying event dataset based on human-machine collaboration. Background Art

[0002] With the popularization of the Internet, social media platforms have become important channels for people to express their opinions. Existing cyberbullying detection datasets have problems such as limited coverage, high annotation costs, insufficient data diversity, and lack of event relevance, making it difficult to comprehensively reflect the complexity and diversity of cyberbullying behavior.

[0003] When constructing a Chinese cyberbullying event dataset with existing technologies, data bias is a significant problem. Due to the diversity and complexity of cyberbullying, data collection may be incomplete, inaccurate, or biased. Such bias may cause the dataset to not truly reflect the actual situation of cyberbullying, thereby affecting subsequent analysis and model training. For example, if certain types of cyberbullying events are lacking in the dataset, or the annotation of certain events is inaccurate, then the model trained based on this dataset may misjudge or miss some of these events; it is impossible to construct a high-quality dataset with wide coverage, accurate annotation, and event association; it is difficult to provide data support for the cyberbullying detection task. Summary of the Invention

[0004] The purpose of the present invention is to overcome the defects of the prior art and provide a method for constructing a Chinese cyberbullying event dataset based on human-machine collaboration. By combining machine-generated pseudo-labels and manual annotation, an event-based Chinese cyberbullying detection dataset (CHNCI) is constructed to solve the problems of limited coverage and high annotation costs of existing datasets.

[0005] The purpose of the present invention is achieved as follows: A method for constructing a Chinese cyberbullying event dataset based on human-machine collaboration includes the following steps:

[0006] 1) Extract comment data related to cyberbullying events from multiple Chinese social media platforms to ensure data diversity and complexity;

[0007] 2) Use three cyberbullying detection methods based on paraphrase, chain of thought, and multi-agent to generate pseudo-labels and corresponding explanatory content, and combine the results of these three detection methods through an integration method;

[0008] 3) Have multiple native Chinese speakers perform manual annotation based on the generated pseudo-labels and explanations to ensure the accuracy of the annotation;

[0009] 4) Construct an event-based Chinese cyberbullying detection dataset CHNCI according to the annotation results and count the basic information of the dataset.

[0010] As a further limitation of the present invention, step 1) specifically includes:

[0011] Step 1.1) Data extraction and collection: Extract comment data related to cyberbullying incidents from multiple Chinese social media platforms; Screen hot events covering multiple fields, where the hot events have a discussion volume of more than 100,000 times within 24 hours and contain at least 5% of controversial content; Crawl comment content, timestamps, anonymous user IDs, and platform source fields through a distributed web crawler.

[0012] Step 1.2) Stratified sampling and sample balancing: Adopt a time-event two-dimensional stratified sampling strategy. In the time dimension, divide the event fermentation period into multiple windows by the hour, and equally spaced samples are extracted to ensure uniform time distribution; In the event dimension, the sample weights are allocated according to the proportion of the original comment volume of the event type to ensure that the sample volume of each type of event is consistent with the actual distribution; Through stratified sampling, representative samples are finally extracted while retaining the time order and event relevance of the comments.

[0013] Step 1.3) Data preprocessing and partitioning: De-duplicate, filter noise, and standardize the original comments; Use the SimHash algorithm to remove duplicate comments, and regular expressions to remove URLs, emojis, and special characters, and eliminate texts with abnormal lengths; Subsequently, unify the timestamp format, text encoding, and simplified and traditional fonts, and construct a metadata matrix containing comment content, time, user, and platform source; Finally, divide the dataset by event, randomly select a certain number of events as the training set, and evenly distribute the remaining events into the validation set and the test set.

[0014] As a further limitation of the present invention, step 2) specifically includes:

[0015] Step 2.1) Pseudo-label generation based on paraphrasing: Use the open-source large language model LLM to perform semantic reconstruction on the input text to generate diverse paraphrased texts; By designing a dynamic prompt template, guide the open-source large language model LLM to generate multiple paraphrased texts, and screen the paraphrased texts with a confidence level higher than the threshold based on cosine similarity; Subsequently, input the paraphrased texts into a pre-trained classifier, and determine the pseudo-labels through majority voting.

[0016] Step 2.2) Pseudo-label and explanation generation based on the chain of thought: Guide the open-source large language model LLM to analyze cyberbullying characteristics through a step-by-step reasoning template; Perform logical consistency verification on the generated reasoning chain, and only retain the conclusions that pass the verification as pseudo-labels; Analyze the background story of the current comment and give an explanation for determining this label.

[0017] Step 2.3) Generation of Pseudo-labels and Explanations Based on Multi-agent: Construct multiple heterogeneous agents, each agent independently generates pseudo-labels; integrate the results through a weighted majority voting mechanism, and the agent weights are dynamically adjusted according to historical accuracy, and the explanation content for determining this label is given;

[0018] Step 2.4) Integration Optimization and Pseudo-label Fusion: Generate the final label by dynamically weighted integration of the pseudo-labels of the three methods in Step 2.1), Step 2.2) and Step 2.3); the weight coefficients are optimized based on the performance of the validation set, and the final label is determined by weighted voting.

[0019] As a further limitation of the present invention, the Step 2.1) specifically includes: using the open-source large language model LLM to perform semantic reconstruction on the input text, generating diversified paraphrased texts and extracting pseudo-labels;

[0020] Paraphrase Generation: Generate k paraphrased versions {X 1 ′, X 2 ′, …, X k ′k} of the input text X, and use a dynamic prompt template to guide the LLM:

[0021]

[0022] Confidence Screening: Calculate the semantic similarity between each paraphrased text and the original text, and retain the texts with a confidence higher than the threshold θ:

[0023] Confidence(X i ′) = cos(Embed(X), Embed(X i ′)), retain Confidence ≥ θ

[0024] where Embed(X), Embed(X i ′) respectively represent the vector representations (also called embedding vectors) of text X and text X i ′;

[0025] Pseudo-label Voting: Input the retained paraphrased texts into a pre-trained classifier, and take the majority voting result as the pseudo-label:

[0026]

[0027] where c = {cyber violence, non-cyber violence}, is the indicator function.

[0028] As a further limitation of the present invention, the Step 2.2) specifically includes: guiding the open-source large language model LLM to generate logically coherent pseudo-labels through a step-by-step reasoning template:

[0029] Chain of Thought Generation: Design the CoT Prompt Template P CoT , and output the reasoning chain S CoT :

[0030]

[0031] Logical Consistency Verification: Extract the key decision nodes in the reasoning chain and calculate the consistency score:

[0032]

[0033] Among them, W ref is the preset brute-force keyword library, and W attack is the list of offensive words. Keep the results with Consistency≥0.7;

[0034] Label Extraction: Extract the final conclusion from the reasoning chain that passes the verification:

[0035] PseudoLabel chain =ExtractLabel(S CoT ).

[0036] As a further limitation of the present invention, step 2.3) specifically includes: generating a pseudo-label through multi-agent collaborative voting:

[0037] Agent Independent Prediction: m heterogeneous agents {Agent 1 ,…,Agent m} respectively generate pseudo-labels:

[0038] y i =Agent i (X)(i = 1,2,…,m);

[0039] Dynamic Weighted Voting: Allocate weights w i according to the historical accuracy of the agents, and generate the final pseudo-label through weighted majority voting:

[0040]

[0041] Among them and ∑w i = 1.

[0042] As a further limitation of the present invention, step 2.4) specifically includes: dynamically fusing the pseudo-labels of three methods:

[0043] Weight Optimization: Maximize the objective function based on the performance of the validation set:

[0044]

[0045] The constraints are α + β + γ = 1 and α, β, γ ≥ 0;

[0046] Among them, α, β, and γ represent the weight parameters of the three methods, satisfying α + β + γ = 1; F1 paraphrase , F1 chain , F1 multi-agent respectively represent the F1 scores of the three methods based on paraphrase, chain of thought, and multi-agent on the validation set;

[0047] Label fusion: Weighted voting generates the final pseudo-label:

[0048]

[0049] Among them, V c paraphrase , V c chain , V c multi-agent respectively represent the number of votes for category c by the three methods based on paraphrase, chain of thought, and multi-agent.

[0050] As a further limitation of the present invention, step 3) specifically includes:

[0051] Step 3.1) Design of the manual annotation process: Multiple native Chinese annotators perform manual annotation based on the machine-generated pseudo-labels and explanations; during the annotation process, the annotators make independent judgments on each comment by combining the reasoning logic of the pseudo-labels and their own semantic understanding; the annotation tool uses an interactive interface, supports highlighting controversial content, adding annotation remarks, and records the annotation time and operation logs;

[0052] Step 3.2) Consistency verification of multiple annotators:

[0053] Majority voting: If at least two annotators reach an agreement, the majority label is adopted;

[0054] Dispute arbitration: If the opinions of all three parties are different, it is submitted to the arbitrator for a final decision;

[0055] Feedback iteration: Incorporate the controversial cases into the update of the annotation guidelines to optimize the subsequent annotation consistency;

[0056] The final annotation result needs to meet the consistency threshold, otherwise re-annotation is required;

[0057] Step 3.3) Inspection and cleaning of annotation quality:

[0058] Random sampling audit: Randomly select 10% of the samples from the dataset, and conduct a secondary verification by an independent audit group to calculate the error rate:

[0059] It is required that ErrorRate ≤ 5%

[0060] Noise elimination: Samples with inconsistent annotations or conflicts with pseudo-labels are directly eliminated or the re-annotation process is triggered;

[0061] Data balance check: Ensure that the distribution of annotated samples for each event type is consistent with the original data to avoid data skew caused by cleaning.

[0062] As a further limitation of the present invention, step 4) specifically includes:

[0063] Step 4.1) Based on the manual annotation results, construct an event-based Chinese cyberbullying detection dataset CHNCI; each data record contains the following fields: a) Comment content: the cleaned text; b) Event ID: the unique identifier associated with the event; c) Label: the final label after manual verification; d) Metadata: timestamp, source platform, user ID, annotator consistency score; e) The data is stored in a structured format and supports multi-dimensional retrieval;

[0064] Step 4.2) Statistically analyze the core metrics of the dataset to verify its comprehensiveness and balance. The core metrics of the dataset include: a) Event scale; b) Total number of comments; c) Label distribution: proportion of violent comments and proportion of non-violent comments; d) Time span: event time distribution; e) Platform distribution.

[0065] The present invention adopts the above technical solutions. Compared with the prior art, the beneficial effects are as follows:

[0066] 1) The present invention proposes a method for constructing an event-based Chinese cyberbullying detection dataset. By combining machine-generated pseudo-labels with manual annotation, a high-quality dataset that covers a wide range, has accurate annotations, and is associated with events is constructed; the advantages of combining machine-generated pseudo-labels with manual annotation are fully utilized to minimize the data annotation cost while ensuring the coverage and quality of the dataset.

[0067] 2) Compared with existing methods that rely on a single annotation method or a single data source, the present invention generates pseudo-labels by integrating multiple cyberbullying detection methods and optimizes them in combination with manual annotation, significantly improving the accuracy and diversity of the dataset. It is an efficient and scalable method for constructing a Chinese cyberbullying detection dataset.

[0068] 3) By associating comment data with specific events, the dataset constructed by the present invention can better support event-driven cyberbullying analysis and prediction, and has important theoretical value and practical significance; the present invention combines event-driven cyberbullying detection with dataset construction for the first time, filling the research gap in this field.

[0069] 4) The method of the present invention has broad application prospects. It can not only be used for network violence detection, but also provide high-quality data support for tasks such as social media content analysis, sentiment analysis, event prediction and social public opinion monitoring, thus promoting research progress in related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 The overall framework diagram of the present invention. DETAILED DESCRIPTION

[0071] like Figure 1 The method for constructing an event-based Chinese network violence detection dataset shown in the figure includes the following steps:

[0072] 1) Extract comment data related to cyber violence incidents from multiple Chinese social media platforms to ensure data diversity and complexity;

[0073] Step 1.1) Data extraction and collection: Extract comment data related to cyber violence incidents from mainstream Chinese social media platforms such as Douyin, Weibo, Xiaohongshu and Bilibili; First, screen hot events with high social attention, covering five major areas of business, entertainment, sports, health and technology, and ensure that the event discussion volume exceeds 100,000 times within 24 hours and contains at least 5% controversial content. Use distributed web crawlers to crawl comment content, timestamps, anonymous user IDs and platform source fields. The data volume of a single event must meet the total number of platform comments of no less than 2,000 to ensure data diversity and event relevance;

[0074] Step 1.2) Stratified sampling and sample balance: Adopt a dual-dimensional stratified sampling strategy of time and event. In the time dimension, divide the event fermentation cycle into multiple windows by hour, and extract samples at equal intervals to ensure uniform time distribution; in the event dimension, assign sample weights according to the proportion of the original number of comments of the event type (such as entertainment, sports), ensuring that the sample size of each type of event is consistent with the actual distribution to avoid data skew. Through stratified sampling, representative samples are finally extracted while retaining the time order of comments and event relevance;

[0075] Step 1.3) Data preprocessing and partitioning: Remove duplicates, filter noise, and standardize the original comments; use the SimHash algorithm to remove duplicate comments, regular expressions to remove URLs, emoticons, and special characters, and remove texts with abnormal lengths (such as less than 5 words or more than 200 words); then unify the timestamp format, text encoding, and simplified and traditional fonts, and build a metadata matrix containing the comment content, time, user, and platform source. Finally, divide the data set by event, randomly select 80% of the events as the training set, and evenly distribute the remaining 20% ​​of the events as the validation set and test set to avoid cross-event data leakage and ensure the reliability of model evaluation.

[0076] 2) Three network violence detection methods based on paraphrasing, chain of thought, and multi-agent are adopted to generate pseudo-labels and corresponding explanatory content, and the results of these three detection methods are combined through an integration method;

[0077] Step 2.1) Pseudo-label generation based on paraphrasing: Use an open-source large language model (LLM) to semantically reconstruct the input text to generate diverse paraphrased texts; by designing a dynamic prompt template (such as "Please rewrite the following text into sentences with the same semantics but different expressions"), guide the open-source large language model (LLM) to generate multiple paraphrased texts, and filter out paraphrased texts with a confidence level higher than a threshold (such as 0.8) based on cosine similarity; subsequently, input the paraphrased texts into a pre-trained classifier and determine the pseudo-labels through majority voting;

[0078] Use an open-source large language model (LLM) to semantically reconstruct the input text, generate diverse paraphrased texts, and extract pseudo-labels; the specific process is as follows:

[0079] a. Paraphrase generation: Generate k paraphrased versions {X 1 ′, X 2 ′,..., X k ′k} of the input text X, and use a dynamic prompt template to guide the LLM:

[0080]

[0081] b. Confidence filtering: Calculate the semantic similarity between each paraphrased text and the original text, and retain the texts with a confidence level higher than the threshold θ:

[0082] Confidence(X i ′) = cos(Embed(X), Embed(X i ′)), retain Confidence ≥ θ

[0083] where Embed(X), Embed(X i ′) respectively represent the vector representations (also known as embedding vectors) of text X and text X i ′;

[0084] c. Pseudo-label voting: Input the retained paraphrased texts into a pre-trained classifier, and take the majority voting result as the pseudo-label:

[0085]

[0086] where c = {cyber violence, non-cyber violence}, is the indicator function.

[0087] Step 2.2) Generation of pseudo-labels and explanations based on the chain of thought: Guide the use of open-source large language model LLM to analyze the characteristics of cyberbullying through a step-by-step reasoning template. For example, require the model to identify offensive words, judge the target, and finally summarize whether it is cyberbullying; perform logical consistency verification on the generated reasoning chain (such as the key judgment node matching rate ≥ 70%), and only retain the conclusions that pass the verification as pseudo-labels to improve the reliability of the reasoning process; analyze the background story of the current comment and give the explanation content for being judged as this label to facilitate the subsequent judgment of the annotator;

[0088] Generate logically coherent pseudo-labels by guiding the open-source large language model LLM through a step-by-step reasoning template:

[0089] a. Generation of the chain of thought: Design the CoT prompt template P CoT , and output the reasoning chain S CoT :

[0090]

[0091] b. Logical consistency verification: Extract the key judgment nodes in the reasoning chain and calculate the consistency score:

[0092]

[0093] where W ref is the preset violent keyword library, and W attack is the list of offensive words. Retain the results with Consistency ≥ 0.7;

[0094] c. Label extraction: Extract the final conclusion from the reasoning chain that passes the verification:

[0095] PseudoLabel chain = ExtractLabel(S CoT ).

[0096] Step 2.3) Generation of pseudo-labels and explanations based on multiple agents: Construct multiple heterogeneous agents (such as based on different LLMs or prompting strategies), and each agent independently generates pseudo-labels; integrate the results through a weighted majority voting mechanism, and the agent weights are dynamically adjusted according to historical accuracy (such as higher-accuracy agents are assigned higher weights), so as to enhance the generalization ability and anti-interference ability of the labels, and give the explanation content for being judged as this label to facilitate the later judgment of the annotator;

[0097] Generate pseudo-labels through multi-agent collaborative voting to improve label robustness:

[0098] a. Independent prediction by agents: m heterogeneous agents {Agent 1 , …, Agent m} respectively generate pseudo-labels:

[0099] y i = Agent i (X)(i = 1, 2, …, m)

[0100] b. Dynamic weighted voting: Assign weights w according to the historical accuracy of the agents i , and generate the final pseudo-label through weighted majority voting:

[0101]

[0102] where and ∑w i = 1.

[0103] Step 2.4) Integration optimization and pseudo-label fusion: Generate the final label by dynamically weighting and integrating the pseudo-labels of the three methods in Step 2.1), Step 2.2), and Step 2.3); the weight coefficients are optimized based on the validation set performance (such as maximizing the F1-score), and the final label is determined by weighted voting; for example, if the method based on paraphrasing performs best in the validation set, a higher weight is assigned (such as α = 0.5), combining the results of the chain of thought (β = 0.3) and multi-agent (γ = 0.2) to ensure the accuracy and stability of the label.

[0104] Dynamically fuse the pseudo-labels of the three methods:

[0105] a. Weight optimization: Maximize the objective function based on the validation set performance:

[0106]

[0107] The constraint is α + β + γ = 1 and α, β, γ ≥ 0;

[0108] where α, β, γ represent the weight parameters of the three methods, satisfying α + β + γ = 1; F1 paraphrase , F1 chain , F1 multi-agent respectively represent the F1 scores of the three methods based on paraphrasing, chain of thought, and multi-agent on the validation set;

[0109] b. Label fusion: Generate the final pseudo-label through weighted voting:

[0110]

[0111] where V c paraphrase , V c chain , V c multi-agent respectively represent the number of votes for class c by the three methods based on paraphrasing, chain of thought, and multi-agent.

[0112] 3) Multiple native Chinese speakers perform manual annotation based on the generated pseudo-labels and explanations to ensure the accuracy of the annotation;

[0113] Step 3.1) Design of the manual annotation process: Multiple native Chinese annotators perform manual annotation based on the pseudo-labels and explanations generated by the machine; Prior to annotation, unified training is required to clarify the criteria for determining cyberbullying (such as offensive language, group discrimination, malicious rumors, etc.), and annotation guidelines and examples are provided; During the annotation process, annotators make independent judgments on each comment by combining the reasoning logic of the pseudo-labels (such as the list of offensive words in the thought chain) and their own semantic understanding; The annotation tool uses an interactive interface, supports highlighting controversial content, adding annotation remarks, and recording the annotation time and operation logs to ensure traceability of the process;

[0114] Step 3.2) Consistency verification among multiple annotators:

[0115] a. Majority voting: If at least two annotators reach an agreement, the majority label is adopted;

[0116] b. Dispute arbitration: If the opinions of all three parties are different, it is submitted to the arbitrator for a final decision;

[0117] c. Feedback iteration: Incorporate controversial cases into the update of the annotation guidelines to optimize the subsequent annotation consistency;

[0118] The final annotation results need to meet the consistency threshold (such as Kappa coefficient κ ≥ 0.85), otherwise re-annotation is required;

[0119] Step 3.3) Inspection and cleaning of annotation quality:

[0120] a. Random sampling and review: Randomly select 10% of the samples from the dataset, and perform secondary verification by an independent review group to calculate the error rate:

[0121] It is required that ErrorRate ≤ 5%

[0122] b. Noise elimination: For samples with inconsistent annotations or conflicts with pseudo-labels (such as the difference between manual and machine labels exceeding 2 rounds of voting), directly eliminate them or trigger the re-annotation process;

[0123] c. Data balance check: Ensure that the distribution of annotated samples for each event type (such as entertainment, sports) is consistent with the original data to avoid data skew caused by cleaning.

[0124] 4) Construct an event-based Chinese cyberbullying detection dataset CHNCI based on the annotation results, and count the basic information of the dataset;

[0125] Step 4.1) Based on the manual annotation results, construct an event-based Chinese cyberbullying detection dataset CHNCI; each data record contains the following fields:

[0126] a. Comment content: The text after cleaning (removing noise and unifying encoding);

[0127] b. Event ID: The unique identifier associated with the event (such as "Entertainment - Cyberbullying Incident of Star A_2023");

[0128] c. Label: The final label after manual verification (Violence / Non-violence);

[0129] d. Metadata: Timestamp, source platform, user ID (anonymized), annotator consistency score.

[0130] e. The data is stored in a structured format (such as JSON or database table), supporting multi-dimensional retrieval by event, time, platform, etc., to ensure data traceability and usability.

[0131] Step 4.2) Statistically analyze the core metrics of the dataset to verify its comprehensiveness and balance. The core metrics of the dataset include:

[0132] a. Event scale: A total of 91 events are included, covering five major types: business, entertainment, sports, health, and technology;

[0133] b. Total number of comments: 227,608 comments, and the range of the number of comments per event is 1,200 - 5,000;

[0134] c. Label distribution: The proportion of violent comments is approximately 18.7%, and the proportion of non-violent comments is 81.3%;

[0135] d. Time span: The event time distribution is from January 2020 to December 2023, covering different social hot cycles;

[0136] e. Platform distribution: Douyin (35%), Weibo (30%), Xiaohongshu (20%), Bilibili (15%).

[0137] To test the effectiveness of the present invention, experiments were conducted in the cyberbullying language detection task and the cyberbullying event prediction task respectively, and the experimental results are as follows:

[0138] Table 1 Accuracy of Cyberbullying Speech Detection

[0139]

[0140] Table 2 Accuracy of Cyberbullying Event Prediction

[0141] Method MAE RMSE GRU 1.4407 2.8429 TCN 1.4169 2.7558 LSTM 1.4039 2.6394 DLinear 1.2746 2.6000 NLinear 1.2512 2.5527 Transformer 1.3952 2.7680 Informer 1.3996 2.7931 Autoformer 1.4463 2.9284 FEDformer 1.2077 2.5259

[0142] The present invention verifies the effectiveness of the proposed method for constructing an event-based Chinese cyberbullying detection dataset through experiments. In the task of detecting cyberbullying remarks, the constructed dataset (CHNCI) is used for testing. As shown in the results of Table 1, the detection methods based on large language models (such as Yi-34B, Qwen-7B, and Llama3-8B) perform excellently in terms of accuracy (Acc) and F1 score (F1s). In particular, Yi-34B achieves the highest accuracy (77.90%) and F1 score (78.47%) on three datasets of different scales, significantly outperforming other traditional methods (such as Bert, HateBert, etc.). This indicates that the methods based on large language models have significant advantages in dealing with complex cyberbullying language detection tasks.

[0143] As shown in Table 2, in the task of predicting cyberbullying events, FEDformer achieves the best results in terms of mean absolute error (MAE) and root mean square error (RMSE) (MAE = 1.2077, RMSE = 2.5259), outperforming other models (such as LSTM, TCN, Transformer, etc.). This indicates that the event-driven method for cyberbullying detection can effectively capture the correlation between events and comments, thus improving the accuracy of event prediction.

[0144] In summary, the present invention constructs a high-quality and widely covered Chinese cyberbullying detection dataset (CHNCI) by combining machine-generated pseudo-labels and manual annotation, and verifies its effectiveness in the tasks of detecting cyberbullying remarks and predicting events. This method not only significantly reduces the data annotation cost, but also improves the diversity and accuracy of the dataset, providing strong data support for the tasks of cyberbullying detection and event prediction, and having important theoretical value and broad application prospects.

[0145] The present invention proposes a method for constructing an event-based Chinese cyberbullying detection dataset. On the one hand, by extracting comment data related to cyberbullying events from multiple Chinese social media platforms, the diversity and complexity of the data are ensured, and the event-driven cyberbullying behavior is associated with the comment data, thus transforming the cyberbullying detection task into an event-driven analysis task. Cyberbullying detection can be regarded as the accurate classification and prediction of comments related to events. On the other hand, the present invention optimizes the data annotation process, uses three cyberbullying detection methods based on paraphrase, chain of thought, and multi-agent to generate pseudo-labels, and combines manual annotation for verification to ensure the accuracy and consistency of the annotation. Through stratified sampling and dataset division strategies, a high-quality and widely covered Chinese cyberbullying detection dataset (CHNCI) is constructed. Finally, based on the constructed dataset, the efficient detection and prediction of cyberbullying events are realized.

[0146] By combining the method of machine-generated pseudo-labels and manual annotation, the present invention significantly reduces the data annotation cost, while improving the coverage and quality of the dataset. By associating comment data with specific events, the dataset constructed by the present invention can better support event-driven cyberbullying analysis and prediction, filling the research gap in this field. In addition, the present invention has broad application prospects, not only can be used for cyberbullying detection, but also can provide high-quality data support for tasks such as social media content analysis, sentiment analysis, event prediction, and social public opinion monitoring, promoting the research progress in related fields.

[0147] The present invention is not limited to the above embodiments. Based on the technical solutions disclosed in the present invention, those skilled in the art can make some substitutions and deformations to some technical features without creative labor according to the disclosed technical content, and these substitutions and deformations are all within the protection scope of the present invention.

Claims

1. A method for constructing a Chinese online violence incident dataset based on human-computer collaboration, characterized in that: The following steps are involved: 1) Extract comment data related to cyber violence incidents from multiple Chinese social media platforms to ensure data diversity and complexity; 2) Three cyber violence detection methods based on paraphrase, thought chain and multi-agent are used to generate pseudo labels and corresponding explanation content, and the results of these three detection methods are combined through an integration method; 3) Multiple native Chinese speakers manually annotate the generated pseudo-labels and explanations to ensure the accuracy of the annotations; 4) Based on the annotation results, an event-based Chinese cyber violence detection dataset CHNCI is constructed, and the basic information of the dataset is statistically analyzed.

2. According to the method for constructing a Chinese network violence incident dataset based on human-computer collaboration in claim 1, it is characterized in that: The step 1) specifically includes: Step 1.1) Data extraction and collection: Extract comment data related to cyber violence incidents from multiple Chinese social media platforms; screen hot events covering multiple fields, where the number of discussions exceeds 100,000 times within 24 hours and contains at least 5% controversial content; crawl comment content, timestamp, anonymous user ID and platform source field through distributed web crawlers; Step 1.2) Stratified sampling and sample balance: A dual-dimensional stratified sampling strategy of time and event is adopted. In the time dimension, the event fermentation cycle is divided into multiple windows by hour, and samples are extracted at equal intervals to ensure uniform time distribution; in the event dimension, sample weights are allocated according to the proportion of the original number of comments of the event type to ensure that the sample size of each type of event is consistent with the actual distribution; through stratified sampling, representative samples are finally extracted while retaining the time order of comments and event relevance; Step 1.3) Data preprocessing and partitioning: deduplicate, filter and standardize the original comments; use the SimHash algorithm to remove duplicate comments, use regular expressions to remove URLs, emoticons and special characters, and remove text with abnormal length; then unify the timestamp format, text encoding and simplified and traditional fonts, and construct a metadata matrix containing the comment content, time, user and platform source; finally, partition the data set by event, randomly select a certain number of events as the training set, and evenly distribute the remaining events as the validation set and test set.

3. According to the method for constructing a Chinese network violence incident dataset based on human-computer collaboration in claim 1, it is characterized in that: The step 2) specifically includes: Step 2.1) Pseudo-label generation based on interpretation: The open source large language model (LLM) is used to semantically reconstruct the input text and generate a variety of interpretation texts; a dynamic prompt template is designed to guide the open source large language model (LLM) to generate multiple interpretation texts, and the interpretation texts with confidence higher than the threshold are selected based on cosine similarity; then, the interpretation texts are input into the pre-trained classifier, and the pseudo-labels are determined by majority voting; Step 2.2) Generate pseudo-labels and explanations based on thought chains: Use the open source large language model LLM to analyze the characteristics of online violence through step-by-step reasoning templates; perform logical consistency checks on the generated reasoning chains, and only retain conclusions that pass the check as pseudo-labels; analyze the background story of the current comment and give the explanation content determined as the label; Step 2.3) Pseudo-label and explanation generation based on multiple agents: construct multiple heterogeneous agents, each of which generates pseudo-labels independently; integrate the results through a weighted majority voting mechanism, dynamically adjust the agent weights based on historical accuracy, and give the explanation content determined as the label; Step 2.4) Integrated optimization and pseudo-label fusion: The pseudo-labels of the three methods in step 2.1), step 2.2) and step 2.3) are integrated through dynamic weighting to generate the final label; the weight coefficient is optimized based on the performance of the validation set, and the final label is determined by weighted voting.

4. According to the method for constructing a Chinese network violence incident dataset based on human-computer collaboration according to claim 3, it is characterized in that: The step 2.1) specifically includes: using an open source large language model LLM to semantically reconstruct the input text, generate a variety of interpretation texts and extract pseudo labels; Paraphrase generation: Generate k paraphrase versions {X1′,X2′,…,X k ′}, use the dynamic prompt template to guide LLM: Confidence screening: Calculate the semantic similarity between each paraphrase text and the original text, and retain texts with confidence higher than the threshold θ: Confidence(X i ′) = cos(Embed(X), Embed(X i ′)), keep Confidence ≥ θ Among them, Embed(X) is the vector representation of text X, Embed(X i ′) is the text X i ′ is a vector representation; Pseudo-label voting: Input the reserved paraphrase text into the pre-trained classifier and take the majority voting result as the pseudo-label: Where c = {cyber violence, non-cyber violence}, is the indicator function.

5. According to the method for constructing a Chinese network violence incident dataset based on human-computer collaboration in claim 3, it is characterized in that: The step 2.2) specifically includes: guiding the open source large language model LLM to generate logically coherent pseudo labels through a step-by-step reasoning template: Thought chain generation: Design CoT prompt template P C o T , output reasoning chain S C o T : Logical consistency check: extract key decision nodes in the reasoning chain and calculate consistency scores: Among them, W ref is the preset violent keyword library, W attack For the offensive word list, the results with Consistency ≥ 0.7 were retained; Label extraction: Extract the final conclusion from the verified reasoning chain: PseudoLabel chain =ExtractLabel(S CoT )。 6. According to the method for constructing a Chinese network violence incident dataset based on human-computer collaboration in claim 3, it is characterized in that: The step 2.3) specifically includes: generating pseudo labels through multi-agent collaborative voting: Agent independent prediction: m heterogeneous agents {Agent1,…,Agent m Generate pseudo labels respectively: y i =Agent i (X)(i=1,2,…,m); Dynamic weighted voting: weight w is assigned based on the historical accuracy of the proxy i , weighted majority voting generates the final pseudo label: in And ∑w i =1.

7. According to the method for constructing a Chinese network violence incident dataset based on human-computer collaboration in claim 3, it is characterized in that: The step 2.4) specifically includes: dynamically fusing the pseudo labels of the three methods: Weight optimization: Maximize the performance of the validation set based on the objective function: The constraints are α+β+γ=1 and α,β,γ≥0; Among them, α, β, and γ represent the weight parameters of the three methods, satisfying α+β+γ=1; F1 paraphrase 、F1 chain 、F1 multi-agent They represent the F1 scores of the three methods based on interpretation, thought chain, and multi-agent on the validation set respectively; Label fusion: weighted voting to generate the final pseudo label: in, Respectively represent the number of votes for category c based on interpretation, thought chain, and multi-agent methods.

8. The method for constructing a Chinese online violence incident dataset based on human-computer collaboration according to claim 1 is characterized in that: The step 3) specifically includes: Step 3.1) Design of manual annotation process: Multiple native Chinese annotators will perform manual annotation based on the pseudo-labels and explanations generated by the machine. During the annotation process, the annotators will make independent judgments on each comment based on the reasoning logic of the pseudo-labels and their own semantic understanding. The annotation tool uses an interactive interface that supports highlighting controversial content, adding annotation notes, and recording annotation time and operation logs. Step 3.2) Multi-annotator consistency check: Majority voting: If at least two annotators agree, the majority label is adopted; Dispute arbitration: If the three parties disagree, the case will be submitted to the arbitrator for final decision; Feedback iteration: Incorporate controversial cases into annotation guide updates to optimize subsequent annotation consistency; The final annotation result must meet the consistency threshold, otherwise it will be re-annotated; Step 3.3) Label quality inspection and cleaning: Random sampling audit: 10% of the samples are randomly selected from the data set, and an independent audit team conducts a second verification and calculates the error rate: Require ErrorRate ≤ 5% Noise removal: remove or trigger a re-labeling process for samples with inconsistent labels or conflicts with pseudo labels; Data balance check: Ensure that the distribution of labeled samples of each event type is consistent with the original data to avoid data skew due to cleaning.

9. The method for constructing a Chinese online violence incident dataset based on human-computer collaboration according to claim 1 is characterized in that: The step 4) specifically includes: Step 4.1) Based on the manual annotation results, construct the event-based Chinese cyber violence detection dataset CHNCI; each data record contains the following fields: a) comment content: cleaned text; b) event ID: unique identifier associated with the event; c) label: final label after manual verification; d) metadata: timestamp, source platform, user ID, annotator consistency score; e) data is stored in a structured format to support multi-dimensional retrieval; Step 4.2) Count the core indicators of the data set to verify its comprehensiveness and balance. The core indicators of the data set include: a) event scale; b) total number of comments; c) label distribution: the proportion of violent comments and the proportion of non-violent comments; d) time span: event time distribution; e) platform distribution.

Images