BERT model-based malicious comment reply detection method and system

Abstract

The invention discloses a malicious comment reply detection method and system based on a BERT model, and the method comprises the steps: picking the historical comment reply data of a user from a platform, and analyzing the data; performing real-time analysis on the newly published comments, and performing data monitoring and early warning; and performing risk response and disposal on the determined malicious comments. The processing capability of the model on complex language structures and long texts is improved, and the generalization capability of the model is enhanced, so that the model can effectively cope with newly appearing malicious comment types, and misinformation and missing report are reduced.

Description

Technical Field

[0001] The present invention relates to the field of user comment reply management, and in particular to a malicious comment reply detection method and system based on a BERT model. Background Art

[0002] The management of user comments has become increasingly important in Internet social media, online forums, and e-commerce platforms. Positive comments can promote product sales and brand image, while negative or malicious comments can damage a company's reputation. Timely monitoring and responding to malicious comments can help companies take quick action to prevent the spread of negative information and protect other users from being misled or harmed, thereby attracting more users to participate in the interaction and increasing user activity.

[0003] Traditional malicious comment detection methods are mostly based on keyword matching or statistical analysis of fixed rules. Although these methods are easy to implement, their accuracy and efficiency are greatly reduced when faced with complex language phenomena and hidden malicious intentions. With the development of deep learning technology.

[0004] Statistical analysis based on keyword matching or fixed rules tends to ignore important context clues. Comment reply is a long conversation process, which requires considering the context. It is not flexible enough and can only recognize pre-set patterns. Once new malicious comments appear, it will not know how to identify them. It is also prone to false positives, misjudging normal comments as malicious. Summary of the invention

[0005] In view of the above problems, the present invention is proposed to provide a method and system for detecting malicious comment replies based on the BERT model, which overcomes the above problems or at least partially solves the above problems.

[0006] According to one aspect of the present invention, a method for detecting malicious comment replies based on a BERT model is provided, and the method for detecting malicious comment replies comprises:

[0007] Extract the comment reply data of users' historical participation from the platform and analyze the data;

[0008] Conduct real-time analysis of newly published comments, conduct data monitoring and early warning;

[0009] Conduct risk response and disposal for identified malicious comments.

[0010] Optionally, extracting the comment reply data of the user's historical participation from the platform and analyzing the data specifically includes:

[0011] Web tracking points capture the comment reply data of users' historical participation from the platform;

[0012] Use regular expressions and other tools to clean the data and remove irrelevant or disruptive content;

[0013] Preprocess the text according to the requirements of the BERT model;

[0014] Break words into segments to handle unseen vocabulary;

[0015] The training data is specially labeled manually to clearly identify malicious comments and non-malicious comments.

[0016] Optionally, the irrelevant or disruptive content specifically includes: URL links, emoticons and irrelevant characters.

[0017] Optionally, the preprocessing of the text specifically includes: word segmentation, encoding, and adding special marks to the text.

[0018] Optionally, the real-time analysis of newly released comments, data monitoring and early warning specifically include: real-time analysis of newly released comments, rapid determination of whether they contain malicious content, and setting a threshold for determining malicious comments based on historical data analysis to ensure that the system can effectively detect malicious comments while minimizing false positives;

[0019] Monitor abnormal behavior patterns in the comment stream and issue warnings.

[0020] Optionally, the abnormal behavior pattern specifically includes: the appearance of a large number of malicious comments in a short period of time.

[0021] Optionally, the risk response and handling of the determined malicious comments specifically include:

[0022] Automatically block, delete or mark malicious comments;

[0023] Send a warning message to the author of the comment;

[0024] Comments that the system is uncertain about or require further confirmation are submitted to the manual review team for judgment;

[0025] Generate reports on malicious comment activities regularly, analyze trends, and provide a basis for subsequent strategy adjustments.

[0026] The present invention also provides a malicious comment reply detection system based on the BERT model, the detection system comprising:

[0027] The data collection and analysis module is used to extract the comment reply data of users' historical participation from the platform and analyze the data;

[0028] Data monitoring and early warning module, used to conduct real-time analysis of newly released comments, data monitoring and early warning;

[0029] The risk response and disposal module is used to respond to and handle risks of identified malicious comments.

[0030] The present invention provides a malicious comment reply detection method and system based on the BERT model, the malicious comment reply detection method comprising: extracting the comment reply data of the user's historical participation from the platform and analyzing the data; performing real-time analysis on the newly released comments, performing data monitoring and early warning; and performing risk response and disposal on the confirmed malicious comments. The model's processing ability for complex language structures and long texts is improved, and the model's generalization ability is enhanced, so that it can effectively deal with new malicious comment types and reduce false positives and false negatives.

[0031] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0033] Figure 1 A business flow chart of a malicious comment reply detection method based on the BERT model provided in an embodiment of the present invention;

[0034] Figure 2 A composition architecture diagram of a malicious comment reply detection system based on the BERT model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0036] The terms "comprises" and "having" and any variations thereof in the description embodiments, claims and drawings of the present invention are intended to cover non-exclusive inclusions, for example, including a series of steps or units.

[0037] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0038] like Figure 1 As shown, a malicious comment reply detection method based on the BERT model includes:

[0039] Extract the comment response data of users' historical participation from the platform and analyze the data; conduct real-time analysis of newly posted comments, conduct data monitoring and early warning; and conduct risk response and disposal for confirmed malicious comments.

[0040] like Figure 1 and Figure 2 As shown, a malicious comment reply detection system based on the BERT model includes: a data collection and analysis module, a data monitoring and early warning module, and a risk response and disposal module;

[0041] Data collection and analysis module: Web tracking points capture the comment reply data of users' historical participation from the platform. Use regular expressions and other tools to clean the data and remove irrelevant or interfering content, such as URL links, emoticons, irrelevant characters, etc. According to the requirements of the BERT model, the text is preprocessed by word segmentation, encoding, adding special tags, etc. Breaking down words into smaller fragments helps to process unseen words. For training data, special manual tags can be performed to clearly identify which comments are malicious and which are non-malicious.

[0042] Data monitoring and early warning module: Real-time analysis of newly released comments to quickly determine whether they contain malicious content. Based on historical data analysis, the threshold for determining malicious comments is set to ensure that the system can effectively detect malicious comments while minimizing false positives. Monitor abnormal behavior patterns in the comment stream, such as the appearance of a large number of malicious comments in a short period of time, and issue warnings for such organized attacks.

[0043] Risk response and disposal module: For confirmed malicious comments, the system can automatically perform operations such as blocking, deleting or marking. Send a warning message to the author of the comment, inform him of the violation, explain the consequences of his posting, understand the reason for the user to persuade him. For comments that the system is uncertain or need further confirmation, submit them to the manual review team for judgment. Generate reports on malicious comment activities regularly, analyze trends, and provide a basis for subsequent strategy adjustments.

[0044] BERT model introduction:

[0045] BERT is a pre-trained model developed by Google. Through unsupervised learning, it can learn a universal language representation, that is, the vector representation of each word. These vector representations can be applied to various natural language processing tasks, such as sentence classification, named entity recognition, question answering, and other tasks.

[0046] By using the Transformer encoder structure, BERT can learn contextual information from both the left and right directions of the text, enabling the model to better understand the semantics of each word in the sentence.

[0047] BERT can be quickly fine-tuned on a variety of downstream tasks through pre-training tasks. When used in specific scenarios, it does not require a large amount of corpus for training, which saves time and is efficient. BERT is such a pre-training model with strong generalization ability. For example, assuming that we have a large amount of Baidu Encyclopedia data, we can use this huge amount of data to train a model with strong generalization ability. When we need to use it in specific scenarios, such as calculating text similarity, we only need to simply modify some output layers, and then use our own data for incremental training and make a slight adjustment to the weights.

[0048] The pre-training phase of BERT includes two tasks, one is Masked Language Model and the other is NextSentence Prediction.

[0049] Masked Language Model:

[0050] Randomly mask some words in the sentence (usually 15%) and predict the original value.

[0051] Maded LM is designed to solve the problem of unidirectional information. The problem with existing language models is that they do not utilize bidirectional information at the same time. For example, ELMO is called a bidirectional LM, but in fact it is a language model composed of two unidirectional RNNs. Due to the relationship between time series, the RNN model predicts that the current word only depends on the words that appeared before, and has no way of knowing the subsequent information.

[0052] So how can we make good use of the semantics of the previous and following words at the same time?

[0053] BERT proposed the Masked Language Model, which randomly masks some tokens in the sentence. The model then predicts the masked words based on the contextual semantics and adjusts the model parameters to maximize the model prediction accuracy.

[0054] The BERT pre-training process imitates the process of learning a language. To accurately understand the semantics of a sentence or a text, we need to learn the contextual relationship and infer the meaning of the missing words from the contextual semantics. Bert's approach simulates the cloze test in English, randomly covering some words and letting the BERT model predict the word, so as to achieve the purpose of learning the semantics of the entire text.

[0055] So how does BERT accomplish “cloze test”?

[0056] Randomly mask 15% of the expected tokens, then predict [MASK]Token, and the final hidden vector corresponding to the masked token is input into the softmax layer on the vocabulary. Although this can indeed train a bidirectional pre-training model, this method has a disadvantage, because during the pre-training process, the random [MASK]Token is fully masked every time, and the MASK information will be remembered during pre-training, but the [MASK]Token has never been seen during fine-tune, resulting in a mismatch between pre-training and fine-tune information.

[0057] In order to solve the mismatch between pre-training and fine-tuning information, BERT does not always replace masked words with actual [MASK]Token. The approach adopted in BERT is: for tokens that need to be MASKed 15%,

[0058] (1) In 80% of cases, it is replaced with [MASK]

[0059] (2) 10% of the time, the token is replaced with a random token.

[0060] (3) 10% of the cases remain unchanged

[0061] For example:

[0062] my dog ​​is hairy → my dog ​​is [MASK] 80% of the selected words are replaced with [MASK]

[0063] my dog ​​is hairy→my dog ​​is apple 10% Replace the selected word with any word

[0064] my dog ​​is hairy→my dog ​​is hairy 10% of the selected words do not change

[0065] [MASK] tells the model in an explicit way, "I won't tell you this word, you have to guess it from the context," to prevent information leakage. If all parts other than [MASK] use the original token, the model will learn, "If the current word is [MASK], infer this word based on the information of other words; if the current word is a normal word, just copy it and input it." In this way, in the fine-tuning stage, all words are normal words, and the model will copy all words without extracting the dependencies between words. Filling in random tokens with a certain probability means that the model is always on guard, and at any token position, it needs to combine the information of the current token with the information inferred from the context. In this way, in normal sentences in the fine-tuning stage, the model will also extract both aspects of information at the same time, because it does not know whether the "normal words" it sees have been tampered with.

[0066] Will adding random tokens confuse the model and prevent it from learning the real semantics? For humans, it is not always possible to complete cloze test correctly, and it is even more difficult to understand if some words in the text are randomly replaced. From a probability perspective, the proportion of random tokens is only 15%*10%=1.5%. If it is expected to be sufficient, this will not affect the performance of the model.

[0067] Next Sentence Prediction:

[0068] Predict whether two sentences are closely connected. The task is to input sentence A and sentence B and determine whether B appears immediately after A.

[0069] In order to train a model that understands sentence relationships, Next Sentence Prediction is proposed, which is to pre-train a binary classification task for next sentence prediction. This task is to randomly select sentence A and sentence B from the corpus before each training, 50% of which are correct adjacent sentences and 50% are randomly selected sentences. This task can achieve an accuracy of 97%-98% in pre-training and can significantly improve the results of QA and NLI.

[0070] Note: The selection of corpus is critical. It is necessary to choose document-level rather than sentence-level corpus, so that it has the ability to abstract long continuous sequence features.

[0071] The model jointly trains the Masked LM task and the Next Sentence Prediction task so that the vector representation of each word / character output by the model can characterize the overall information of the input text (single sentence or sentence pair) as comprehensively and accurately as possible, providing better initial values ​​of model parameters for subsequent fine-tuning tasks.

[0072] enter:

[0073] BEBERT uses a combination of three embedding vectors as input representation:

[0074] 1.Token Embedding: This part converts each word in the input text into a vector of fixed size. Usually, this is done by looking up a pre-trained embedding table where each word corresponds to a point in a multidimensional space.

[0075] 2. Segment Embedding: BERT can handle a variety of tasks, including single sentence classification and sentence pair tasks (such as question answering and natural language inference). In order to distinguish between two different sentences in a sentence pair, BERT introduces paragraph embedding. For example, when processing a sentence pair, each word in the first sentence will be attached with a paragraph embedding of "A", and each word in the second sentence will be attached with a paragraph embedding of "B".

[0076] 3. Position Embedding: Since the Transformer architecture that BERT is based on is not cyclic, it does not have the ability to process sequence data. Therefore, in order to enable the model to understand the order of words in a sentence, BERT adds position embedding. Position embedding is pre-trained and can encode the information of each position and add it to the word embedding, so that the model can understand the order information of the text.

[0077] By adding these three embeddings, BERT can generate a comprehensive representation for each input word, which contains the meaning of the word itself, its role in the sentence (for example, which sentence it belongs to), and its position in the sentence. This comprehensive representation is the basis for input to the subsequent layers of the BERT model (i.e., the multi-layer Transformer encoder) for further feature extraction and learning.

[0078] Output:

[0079] The BERT model adds an output layer based on the specific downstream task. The structure of this output layer varies depending on the task. For example, in a text classification task, the output layer may be a simple softmax classifier that predicts the category of the input text. In a question-answering task, the output layer may be a softmax classifier that generates start and end positions. In sequence labeling tasks such as named entity recognition, the output layer usually uses a model structure such as conditional random fields (CRF). The purpose of this layer is to map the output of the BERT encoder to the output space of a specific task, and perform appropriate loss calculations and parameter optimization depending on the task.

[0080] Depending on the requirements of different tasks, BERT has different output layer designs:

[0081] 1. Classification tasks: For tasks such as text classification, BERT can use the output of a special [CLS] tag as the representation of the entire sentence, and then connect a fully connected layer (usually a softmax layer) to predict the category the text belongs to. This method can encode the semantic information of the entire sentence into the vector of the [CLS] tag and use it for classification.

[0082] 2. Sequence labeling tasks: For sequence labeling tasks such as part-of-speech tagging and named entity recognition, BERT can directly add a fully connected layer (usually a softmax layer) to the output of each word to predict the label of each word. In this case, the output of each word can be used to predict the label of the word, thereby completing the sequence labeling task.

[0083] 3. Question answering system: For question answering tasks, such as extractive question answering in reading comprehension (such as SQuAD tasks), BERT can use a special output layer design that combines the prediction of the start and end positions. Usually, when BERT predicts the start and end positions, it connects a fully connected layer (usually a softmax layer) to each position to predict the start and end positions of the answer.

[0084] 4. Text matching tasks: For text matching tasks such as natural language inference and similarity matching, BERT uses a variety of methods. A common method is to concatenate the representations of two sentences and then classify them through a fully connected layer to determine the relationship between the two sentences (such as implication, contradiction, neutrality).

[0085] Beneficial effects: Malicious comment detection based on the BERT model improves the model's ability to process complex language structures and long texts, and enhances the model's generalization ability, enabling it to effectively respond to newly emerging malicious comment types and reduce false positives and negatives.

[0086] The above specific implementation methods further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting malicious comment replies based on the BERT model. It is characterized in that The malicious comment reply detection method comprises: Extract the comment reply data of users' historical participation from the platform and analyze the data; Conduct real-time analysis of newly published comments, conduct data monitoring and early warning; Conduct risk response and disposal for identified malicious comments.

2. According to the method for detecting malicious comment replies based on the BERT model according to claim 1, It is characterized in that The data of user historical comments and responses extracted from the platform and analyzed specifically includes: Web tracking points capture the comment reply data of users' historical participation from the platform; Use regular expressions and other tools to clean the data and remove irrelevant or disruptive content; Preprocess the text according to the requirements of the BERT model; Break words into segments to handle unseen vocabulary; The training data is specially labeled manually to clearly identify malicious comments and non-malicious comments.

3. According to the method for detecting malicious comment replies based on the BERT model according to claim 2, It is characterized in that The irrelevant or disruptive content specifically includes: URL links, emoticons and irrelevant characters.

4. According to claim 2, a method for detecting malicious comment replies based on a BERT model, It is characterized in that The text preprocessing work specifically includes: word segmentation, encoding, and adding special marks to the text.

5. According to the method for detecting malicious comment replies based on the BERT model according to claim 1, It is characterized in that The real-time analysis of newly released comments, data monitoring and early warning specifically include: Analyze newly published comments in real time to quickly determine whether they contain malicious content. Set the threshold for judging malicious comments based on historical data analysis to ensure that the system can effectively detect malicious comments while minimizing false positives. Monitor abnormal behavior patterns in the comment stream and issue warnings.

6. According to the method for detecting malicious comment replies based on the BERT model according to claim 5, It is characterized in that The abnormal behavior pattern specifically includes: the appearance of a large number of malicious comments in a short period of time.

7. According to the method for detecting malicious comment replies based on the BERT model according to claim 1, It is characterized in that The risk response and disposal of the confirmed malicious comments specifically include: Automatically block, delete or mark malicious comments; Send a warning message to the author of the comment; Comments that the system is uncertain about or require further confirmation are submitted to the manual review team for judgment; Generate reports on malicious comment activities regularly, analyze trends, and provide a basis for subsequent strategy adjustments.

8. A malicious comment reply detection system based on BERT model, It is characterized in that The detection system comprises: The data collection and analysis module is used to extract the comment reply data of users' historical participation from the platform and analyze the data; Data monitoring and early warning module, used to conduct real-time analysis of newly released comments, data monitoring and early warning; The risk response and disposal module is used to respond to and handle risks of identified malicious comments.

Images