A method and system for detecting abusive comments

Abstract

The present application provides a method and system for detecting abusive comments, which relates to the fields of artificial intelligence and language processing technology. A multi-level vector representation is selected, and a secondary pre-training model BERT whose parameters have been optimized in a large number of lexicons is selected to encode the comments. It can not only ensure that the encoder can fully represent the final semantic representation, but also further improve the accuracy and recall rate of the detection model. The present invention fully combines the advantages of strategy detection and algorithm detection, and through the clever combination of the two, reduces labor costs, and maximizes the accuracy and effectiveness of model detection under the premise of minimum computing power.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and language processing technology, and specifically to a method and system for detecting abusive comments. Background Art

[0002] Although the keyword filtering method can intercept some abusive comments to a certain extent, there are two major problems: when the content of the abusive vocabulary is small, the keyword coverage will be insufficient, resulting in a decrease in the recall rate of the detection model; when the content of the abusive vocabulary is rich, it means that many words related to the abusive vocabulary will be included, resulting in that any comments that hit the content in the abusive vocabulary will be filtered out, resulting in a decrease in the accuracy of the model detection. In addition, the construction of the abusive vocabulary also has the disadvantages of wasting manpower and inefficient feature selection. Although the neural network classification model works well in the training of the data set, it is found that when the test data set has errors in the effect of the model representation, the final detection model will have low accuracy.

[0003] Based on the above analysis, it is particularly important to propose a model that can effectively detect abusive comments. Summary of the invention

[0004] Aiming at the shortcomings of the existing abusive comment detection model, the present invention combines the advantages of keyword filtering strategy and neural network algorithm, and proposes a method and system for detecting abusive comments, which relates to the fields of artificial intelligence and language processing technology. The present invention uses semantic representation at the character level and word level to perform multi-level modeling on the underlying model, thereby mining the deep features of the text, effectively improving the accuracy of abusive comments on the Internet, and reducing the cost of manual feature extraction and the cost of maintaining and updating the abusive vocabulary in the later stage.

[0005] In order to solve the above technical problems, this application provides the following technical solutions:

[0006] In a first aspect, the present application provides a method for detecting abusive comments, comprising:

[0007] Get the comment text to be detected;

[0008] Filtering the comment text to be detected;

[0009] Input the filtered comment text to be detected into the preset abusive comment detection model to obtain the detection result of the comment text to be detected;

[0010] The abusive comment detection model is formed by training word vectors and character vectors corresponding to multiple historical comment texts.

[0011] Furthermore, before inputting the comment text to be detected into the preset abusive comment detection model, the method further includes:

[0012] The review text to be detected is truncated according to a preset maximum length to obtain multiple words to be detected;

[0013] Performing preliminary detection on the plurality of words to be detected according to a preset abusive vocabulary list;

[0014] If at least one of the multiple words to be detected matches an abusive word in the abusive vocabulary list, determining that the result of the preliminary detection is that the comment text to be detected is an abusive comment;

[0015] Correspondingly, the filtering of the comment text to be detected is specifically as follows:

[0016] Filter out the comment texts to be tested that are not abusive comments in the preliminary test results;

[0017] The step of inputting the comment text to be detected into the preset abusive comment detection model is as follows:

[0018] The result of the preliminary detection is that the text of the comment to be detected which is not an abusive comment is input into the preset abusive comment detection model.

[0019] Furthermore, the preliminary detection of the plurality of words to be detected according to the preset abusive vocabulary list includes:

[0020] Performing a word embedding operation on each of the to-be-detected words and each of the abusive words respectively to obtain a first word vector corresponding to each of the to-be-detected words and a second word vector corresponding to each of the abusive words;

[0021] For each first word vector, calculate the cosine value of the angle between it and each second word vector;

[0022] Correspondingly, when the cosine value of the angle exceeds a preset threshold, it is considered that the to-be-detected word corresponding to the first word vector matches the abusive word corresponding to the second word vector.

[0023] Furthermore, the step of training the abusive comment detection model includes:

[0024] Collecting a plurality of historical review texts, and filtering each historical review text according to the abusive vocabulary list to obtain training historical review texts that do not hit the abusive vocabulary list;

[0025] The training historical review text is processed using a convolutional neural network with randomly initialized weights and a Chinese corpus to obtain corresponding word vectors and character vectors;

[0026] Pre-train the preset classification model so that it can learn the deep semantic information of the comment text;

[0027] The pre-trained classification model is trained again using the word vector and character vector to obtain the abusive comment detection model.

[0028] Furthermore, the convolutional neural network with randomly initialized weights and the Chinese corpus are used to process the training historical comment text to obtain corresponding word vectors and character vectors, including:

[0029] Separating the training history review text into characters to obtain corresponding multiple words;

[0030] Encoding the multiple words using a convolutional neural network with randomly initialized weights to obtain word vectors corresponding to the training historical comment text; and

[0031] Performing word segmentation processing on the training history review text to obtain corresponding multiple words;

[0032] The multiple words are represented according to a preset Chinese corpus to obtain word vectors corresponding to the training historical comment text.

[0033] Furthermore, the using the word vector and the character vector to perform secondary training on the pre-trained classification model includes:

[0034] Generate a corresponding context semantic vector according to the word vector and character vector corresponding to the training historical comment text;

[0035] Inputting the context semantic vector corresponding to the training historical comment text and the preset true classification result into the pre-trained classification model to obtain an updated classification model;

[0036] Repeat the above steps to train and update the classification model until the updated classification model converges;

[0037] A neural network model is constructed according to the converged classification model parameters to obtain the abusive comment detection model.

[0038] Furthermore, the step of generating a corresponding context semantic vector according to the word vector and character vector corresponding to the training historical comment text includes:

[0039] Add a beginning indicating word and an ending indicating word at the beginning and the end of each training history comment text, respectively, to obtain the input text corresponding to each training history comment text;

[0040] The word vectors and character vectors corresponding to the training history comment texts and the input text are input into an encoder to obtain context semantic vectors of each training history comment text.

[0041] Further, the pre-trained classification model includes two layers of feed-forward neural network linear layers;

[0042] The context semantic vector corresponding to the training historical comment text and the preset real classification result are inputted into the pre-trained classification model to obtain an updated classification model, including:

[0043] Inputting the context semantic vector corresponding to the training historical comment text and the preset classification result into the first-layer forward neural network linear layer for dimensionality reduction processing, so that the first-layer forward neural network linear layer outputs the dimensionality reduction feature vector of the context semantic vector;

[0044] Inputting the dimension-reduced feature vector of the context semantic vector into the second-layer feed-forward neural network linear layer, so that the second-layer feed-forward neural network linear layer outputs the detection result of the training history comment text;

[0045] The parameters of the classification model are updated according to the detection results and the corresponding true classification results to obtain an updated classification model.

[0046] Furthermore, the filtered comment text to be detected is input into a preset abusive comment detection model to obtain a detection result of the comment text to be detected, including:

[0047] A convolutional neural network with randomly initialized weights and a Chinese corpus are used to process the review text to be detected, and the corresponding word vectors and character vectors are obtained;

[0048] Generate the corresponding context semantic vector based on the word vector and character vector of the comment text to be detected;

[0049] The context semantic vector of the comment text to be detected is input into the abusive comment detection model to obtain the detection result of the comment text to be detected.

[0050] Furthermore, obtaining the comment text to be detected includes:

[0051] Get the complete sentence of the comment to be tested;

[0052] Removing non-text characters from the complete sentence of the comment to be detected to obtain the text of the comment to be detected;

[0053] The non-text characters include emoticons, labels, and special characters.

[0054] In a second aspect, the present application provides a detection system for abusive comments, comprising:

[0055] A text acquisition module is used to obtain the comment text to be detected;

[0056] A text filtering module, used for filtering the comment text to be detected;

[0057] A text detection module, used to input the filtered comment text to be detected into a preset abusive comment detection model to obtain a detection result of the comment text to be detected;

[0058] The abusive comment detection model is formed by training word vectors and character vectors corresponding to multiple historical comment texts.

[0059] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any method for detecting abusive comments provided in the present application is implemented.

[0060] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the computer program implements any method for detecting abusive comments provided in the present application.

[0061] It can be seen from the above technical solution that the present application provides a method and system for detecting abusive comments, which selects a multi-level vector representation, and selects the secondary pre-training model BERT whose parameters have been optimized in a large number of lexicons to encode the comments. It can not only ensure that the encoder can fully represent the final semantic representation, but also further improve the accuracy and recall rate of the detection model. The present invention fully combines the advantages of strategy detection and algorithm detection, and through the clever combination of the two, reduces labor costs, and maximizes the accuracy and effectiveness of model detection under the premise of minimum computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0063] Figure 1 It is a flowchart of a method for detecting abusive comments in an embodiment of the present application.

[0064] Figure 2 It is another flow chart of the method for detecting abusive comments in the embodiment of the present application.

[0065] Figure 3 It is another flow chart of the method for detecting abusive comments in the embodiment of the present application.

[0066] Figure 4It is another flow chart of the method for detecting abusive comments in the embodiment of the present application.

[0067] Figure 5 It is another flow chart of the method for detecting abusive comments in the embodiment of the present application.

[0068] Figure 6 Another flowchart of the method for detecting abusive comments in the embodiment of the present application is shown in FIG.

[0069] Figure 7 It is another flow chart of the method for detecting abusive comments in the embodiment of the present application.

[0070] Figure 8 It is another flow chart of the method for detecting abusive comments in the embodiment of the present application.

[0071] Fig. 9 It is another flow chart of the method for detecting abusive comments in the embodiment of the present application.

[0072] Fig.10 It is a structural diagram of a detection system for abusive comments in an embodiment of the present application.

[0073] Fig.11 It is a schematic diagram of the structure of the abusive comment detection model in the embodiment of the present application.

[0074] Fig.12 It is a schematic diagram of the operation flow of the abusive comment detection system in the embodiment of the present application.

[0075] Fig.13 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0077] This application can be used in the field of artificial intelligence, and of course, can also be used in other fields, and this application does not limit it.

[0078] At present, although the keyword filtering method can block some abusive comments to a certain extent, there are two major problems: when the content of the abusive vocabulary is small, it will lead to insufficient keyword coverage, resulting in a decrease in the recall rate of the detection model.

[0079] Based on this, the present application provides an embodiment of a method for detecting abusive comments, see Figure 1 ,include:

[0080] Step S100: Obtain the comment text to be detected;

[0081] Step S200: filtering the comment text to be detected;

[0082] Step S300: inputting the filtered comment text to be detected into a preset abusive comment detection model to obtain a detection result of the comment text to be detected; wherein the abusive comment detection model is formed by training word vectors and character vectors corresponding to multiple historical comment texts.

[0083] It is understandable that the BERT model is a language model based on bidirectional Transformer released by the Google AI team, and the corresponding paper is BERT: Pre-train of Deep Bidirectional Transformers for Language Understanding. This type of model is a transferable model, which mainly combines the pre-trained model with the downstream task model, and is a natural model that can support text classification tasks. Google has trained 7 different model files for different downstream tasks according to the number of Transformer layers and the case settings. The abusive comment detection model of this application can be a BERT neural network model that has been trained twice, and the pre-trained model BERT is trained twice. The specific method is as follows: First, load the initial parameters of the model to ensure the maximum learning representation ability of BERT. Since BERT is mostly trained and learned on large English data sets, in order to better adapt to the downstream tasks of the Chinese abusive evaluation data set, a static masking method is adopted, that is, a part of each data is randomly replaced with a "MASK" token, and then it is allowed to predict the next sentence, and the model with the smallest loss value is saved. In this process of continuous secondary pre-training and prediction, the parameters will be continuously adjusted so that it can fully learn the characteristics of the current data set and be more suitable for downstream tasks.

[0084] We selected a multi-level vector representation and the secondary pre-trained model BERT, whose parameters have been optimized in a large number of lexicons, to encode the comments. This not only ensures that the encoder can fully represent the final semantic representation, but also further improves the accuracy and recall of the detection model. The present invention fully combines the advantages of strategy detection and algorithm detection, and through the clever combination of the two, reduces labor costs, and maximizes the accuracy and effectiveness of model detection under the premise of minimum computing power.

[0085] Semantic representation refers to converting human language into data form so that neural networks can further process it. Since language is composed of words, it can be traced back to the representation of words.

[0086] In some embodiments, the word is represented in one-hot form, that is, all the words are sorted, and the ordinal position corresponding to the word is represented by 1 using a high-dimensional sparse matrix, and the other positions are 0. However, this method has a large dimension, and the words are independent of each other, and it is impossible to represent the semantic information between words. For example, the three words "emperor", "queen" and "cat", it can be clearly seen that the first two words have a certain semantic connection. However, the semantic connection between "cat" and the first two words is not great.

[0087] In addition, in some embodiments, the word representation of the present invention can also adopt methods about distributed representation, including matrix-based, cluster-based, and neural network-based distributed representations. Currently, the more mature distributed representation based on neural networks, a typical representation is word2vec, the core of which is modeling based on the representation of context and the relationship between context and target words. In addition, there are some commonly used word vector tools: CNN, Glove, fasttext, ELMO, etc., all of which have shown excellent performance on different tasks.

[0088] In one embodiment of the method for detecting abusive comments provided in the present application, a preferred method for filtering the comment text to be detected is provided. Figure 2 Before inputting the comment text to be detected into the preset abusive comment detection model, the abusive comment detection method further includes:

[0089] Step S201: truncating the review text to be detected according to a preset maximum length to obtain a plurality of words to be detected;

[0090] Step S202: performing preliminary detection on the plurality of to-be-detected words according to a preset abusive vocabulary list; specifically, detecting that at least one word among the plurality of to-be-detected words matches an abusive word in the abusive vocabulary list;

[0091] Step S203: Determine whether the result of the preliminary detection is that the comment text to be detected is an abusive comment; if so, execute step 204; if not, execute step S300;

[0092] Step S300: inputting the filtered comment text to be detected into a preset abusive comment detection model;

[0093] Step S204, outputting the detection results corresponding to the comments to be detected, wherein the detection results include abusive comments and non-abusive comments.

[0094] Correspondingly, the filtering of the comment text to be detected is specifically as follows:

[0095] Filter out the comment texts to be tested that are not abusive comments in the preliminary test results;

[0096] The step of inputting the comment text to be detected into the preset abusive comment detection model is as follows:

[0097] The result of the preliminary detection is that the text of the comment to be detected which is not an abusive comment is input into the preset abusive comment detection model.

[0098] It is understandable that the purpose of creating an abusive vocabulary list is to be able to use a feature-based approach to quickly find words with obvious abusive meanings contained in the text, and then judge whether the comment text to be detected is an abusive comment, so as to label it. The abusive vocabulary list includes words with obvious abusive meanings, that is, unambiguous abusive words. For comment texts containing abusive words, the detection results can be obtained more quickly by matching the abusive vocabulary list, and the detection efficiency is higher. Therefore, this embodiment combines the two methods of abusive vocabulary list matching and model detection, which can not only achieve rapid detection when the comment to be detected contains abusive words, but also avoid the omission of comments with implicit abusive meanings as much as possible, which helps to improve the accuracy and detection efficiency of abusive comment detection, while reducing the computational pressure of the abusive comment detection model.

[0099] Among them, the abusive vocabulary list can be formed through the following steps: using python crawler technology to crawl the text data of real Internet comments; using rule-based methods to roughly filter the data, remove emoticons, tags, special characters, etc., to ensure that each comment is pure text data; finally, set the maximum length, truncate each comment, and obtain multiple words; collect words with obvious abusive meanings to establish an abusive vocabulary list. Here, "words with obvious abusive meanings" refer to unambiguous abusive words.

[0100] In one embodiment, see Figure 3 Step S202, performing preliminary detection on the plurality of words to be detected according to a preset abusive comment table, including:

[0101] Step S2021, performing a word embedding operation on each of the to-be-detected words and each of the abusive words, respectively, to obtain a first word vector corresponding to each of the to-be-detected words and a second word vector corresponding to each of the abusive words;

[0102] Step S2022, for each first word vector, respectively calculate the cosine value of the angle between it and each second word vector; the cosine value is used to represent the similarity between the first word vector and the second word vector.

[0103] Correspondingly, when the cosine value of the angle exceeds a preset threshold, that is, when the similarity between the first word vector and the second word vector exceeds the preset threshold, it is considered that the word to be detected corresponding to the first word vector matches the abusive word corresponding to the second word vector.

[0104] It can be understood that the review text to be tested is first segmented using the Jieba segmentation tool to obtain the sentence representation T'={t1, t2, ..., t N When each character in the review text to be detected cannot form a regular word with its surrounding characters, the character is segmented into single characters. When the character and its surrounding characters form a regular word, the regular phrase is segmented into words. The single characters and single words are used as the words to be detected in step S021. Then, for each word to be detected in T', i and each insult word m in the insult vocabulary M j Perform word embedding representation of Glove and obtain vector representation of two words:

[0105]

[0106]

[0107] Then use the cosine of the angle between them to calculate the similarity:

[0108]

[0109] If the similarity is greater than a preset threshold value k, it means that the word to be detected matches the abusive word, that is, the comment text to be detected is marked as an abusive comment text. Here, the preset threshold value can be set to k=0.8, for example.

[0110] In one embodiment of the method for detecting abusive comments provided in the present application, a preferred method for training an abusive comment detection model is provided, see Figure 4 , the specific steps of training word vectors and character vectors corresponding to multiple historical comment texts include:

[0111] Step S401, collecting multiple historical comment texts, and filtering each historical comment text according to the abusive comment table to obtain training historical comment texts that do not hit the abusive comment table;

[0112] Specifically, the filtering method can be implemented with reference to the steps in step S201 and step S202, and all the historical comment texts obtained are filtered to obtain historical comment texts that do not contain abusive words. It can be understood that the present application divides all the filtered historical comment texts into a training set, a validation set, and a test set according to 8:1:1, and the training set is the training historical comment text of the present application.

[0113] Step S402, using a convolutional neural network with randomly initialized weights and a Chinese corpus to process the training historical comment text to obtain corresponding word vectors and character vectors;

[0114] Step S403, pre-training the preset classification model so that it can learn the deep semantic information of the comment text.

[0115] Through multiple comparisons, this application selects the BERT model, which has the best performance in classification tasks, as the preset classification model. This application chooses to load the pre-trained model to pre-train the classification model so that it can learn feature representation to the maximum extent. The pre-trained classification model can learn the deep semantic information of the comment text.

[0116] Step S404, using the word vector and character vector to perform secondary training on the pre-trained classification model to obtain the abusive comment detection model.

[0117] The abusive comment detection model of the present application is trained on the basis of a classification model that can learn the deep semantic information of the comment text. Therefore, the abusive comment detection model can perform speech analysis on the comment text to be detected, determine whether the comment text to be detected has abusive meaning, and thus determine whether the comment to be detected is an abusive comment.

[0118] In one embodiment, if Figure 5 As shown, step S402, a convolutional neural network with randomly initialized weights and a Chinese corpus are used to process the training historical comment text to obtain corresponding word vectors and character vectors, including:

[0119] Step S4021, performing character separation on the training history comment text to obtain corresponding multiple single words;

[0120] Step S4022, encoding the multiple words using a convolutional neural network with randomly initialized weights to obtain word vectors corresponding to the training historical comment text;

[0121] Specifically, step S4021 and step S4022 define the process of generating word vectors. In the word-level representation, word segmentation is not required. The text T is directly separated into individual characters using CharTokenizer and encoded using one-hot. The token representation of the text is obtained as 〖t〗^c={t_1^c,t_2^c,…,t_N^c}∈R^N, where N represents the number of words.

[0122] This application uses a convolutional neural network (CNN) with randomly initialized weights to obtain an effective feature-level embedding representation for each word through multiple convolutional layers, and performs feature selection and dimensionality reduction through max-pooling in the pooling layer, retaining only important information, thereby obtaining a fixed-size vector representation of each word for word vector representation. Since bert_base_uncase of the BERT model is needed, the latitude is uniformly set to 768 when performing embedded representation. Therefore,

[0123] Step S4023, performing word segmentation processing on the training history comment text to obtain corresponding multiple words;

[0124] Step S4024, representing the multiple words according to a preset Chinese corpus to obtain word vectors corresponding to the training historical comment text.

[0125] Specifically, step S4023 and step S4024 define the process of generating word vectors. At the word level, the jieba word segmentation developed by the HIT NLP team is used to segment the text of each comment, and the token representation after segmentation is obtained as t^b={t_1^b, t_2^b, …, t_M^b}∈R^M, where M represents the number of words after segmentation in the text; the encodings 〖t〗^c and t^b are input into the embedded representation layer to obtain the corresponding vectors. Then s uses SpaceTokenizer to separate the text by spaces, and then uses google_zh_vocab trained on the Chinese corpus in BERT to obtain the fixed vector representation of each segmentation for word vectorization.

[0126] In one embodiment, see Figure 6 , step S404, using the word vector and character vector to perform secondary training on the pre-trained classification model, including:

[0127] Step S4041, generating a corresponding context semantic vector according to the word vector and character vector corresponding to the training historical comment text;

[0128] Specifically, the character vector plus the word vector feature can not only learn the contextual semantic environment, but also learn the internal structural features of the word. The Highway network is used to fuse the character-level embedding representation with the word-level embedding representation to obtain the contextual representation word. The context representation words are encoded to obtain the context semantic vector.

[0129] Step S4042, inputting the context semantic vector corresponding to the training historical comment text and the preset real classification result into the pre-trained classification model to obtain an updated classification model;

[0130] Specifically, the classification model includes a classification output layer, into which the context semantic vector corresponding to each training historical comment text and its corresponding true classification result are input, and the classification output layer outputs the detection results of the historical sentences in the training data set; the classification model also includes a loss function calculation layer, which uses the cross-entropy loss function to calculate:

[0131]

[0132] Step S4043, determining whether the updated classification model has reached convergence; if so, executing step S4044; if not, repeating the above steps S4041 to S4043;

[0133] Specifically, when judging whether the updated classification model has reached convergence, the validation set is used for testing, and the parameters of the classification model with the best performance on the validation set are saved. When judging, the test set is layered and embedded, and then input into the classification model with the best performance. Finally, the classification accuracy is used to evaluate the performance of the model:

[0134]

[0135] Among them, TP represents the number of samples predicted to be positive and actually positive, TN represents the number of samples predicted to be negative and actually negative, FP represents the number of samples predicted to be positive and actually negative, and FN represents the number of samples predicted to be negative and actually positive.

[0136] Step S4044, constructing a neural network model according to the converged classification model parameters to obtain the abusive comment detection model.

[0137] In one embodiment, see Figure 7 , step S4041, generating a corresponding context semantic vector according to the word vector and character vector corresponding to the training historical comment text, including:

[0138] Step S40411, adding a beginning indicating word and an ending indicating word to the beginning and the end of each training history comment text, respectively, to obtain an input text corresponding to each training history comment text;

[0139] Step S40412, input the word vectors and character vectors corresponding to the training history comment texts and the input text into an encoder to obtain the context semantic vectors of each training history comment text.

[0140] It can be understood that the abusive comment detection model includes an encoding layer, which uses the vectors corresponding to the words and characters in each historical sentence and the historical abusive sentence in the encoding layer to encode the context semantic vector, specifically including: the obtained hierarchical embedding representation {e1, e2, ..., e N} as input, and encode it using the BERT model obtained after the second pre-training. The BERT model consists of a 12-layer bidirectional transformer encoder. Before inputting the model, it is necessary to add a representation [CLS] that can indicate the beginning of the text before and after the sentence. And a representation that can indicate the end of the text [SEP]. The input form of the BERT model S0:

[0141] T=[CLS]+sentence+[SEP]

[0142] S0=E T +E P +E S

[0143] in,

[0144] After obtaining the embedded representation of the model, it is encoded using K consecutive bidirectional transformer modules:

[0145] S i = Transformer(S I-1 )

[0146] Among them, S i represents the output of the i-th layer transformer, i∈[1,12]. Each Transformer contains a Maskedmulti-head attention layer and a feed-forward layer. The multi-head attention layer performs the self-attention process h times and then combines the outputs. The specific calculation is as follows:

[0147]

[0148] head i =Attention(Q, K, V)

[0149] MultiHead(Q,K,V)=[head1,…,head h ]W O

[0150] Among them, Q, K, and V are obtained by multiplying the input word vector by the corresponding weight matrix. W O is a learnable parameter, h=12, indicating the number of layers of Attention, Among them, d model =768, which is the default parameter setting of BERT. Q, K, and V are transformed through a Linear layer, and then Q and K are matrix-multiplied for dimension scaling, and then softmax is performed to obtain the weight matrix, which is multiplied by V to obtain the output of each head.

[0151] The specific calculation of the Feed-forward layer is as follows:

[0152]

[0153]

[0154] Among them, O i represents the output representation after the i-th Multi-head attention layer, ρ and μ are learnable parameters. Finally, the output of each transformer layer is obtained.

[0155]

[0156] Total output And extract the encoding representation of the CLS of each layer at the end

[0157] H={H 1 , H 2 , …, H h}

[0158]

[0159] In one embodiment, see Figure 8 , the pre-trained classification model includes two layers of forward neural network linear layers; step S4042, inputting the context semantic vector corresponding to the training historical comment text and the preset real classification result into the pre-trained classification model to obtain an updated classification model, including:

[0160] Step S40421, inputting the context semantic vector corresponding to the training historical comment text and the preset classification result into the first layer of the forward neural network linear layer for dimensionality reduction processing, so that the first layer of the forward neural network linear layer outputs the reduced dimensionality feature vector of the context semantic vector;

[0161] Step S40422, inputting the dimension-reduced feature vector of the context semantic vector into the second-layer feed-forward neural network linear layer, so that the second-layer feed-forward neural network linear layer outputs the detection result of the training history comment text;

[0162] Step S40423, updating the parameters of the classification model according to the detection result and the corresponding true classification result to obtain an updated classification model.

[0163] After obtaining the encoded context semantic vector, the semantic vector of [CLS] is selected by default as the representation of the training historical comment text. It is used as input to pass through two layers of forward neural network linear layers, and the size of the output sample is specified for dimensionality reduction through linear transformation. The high-dimensional features are reduced to low-dimensional features by multiplication with the weight matrix, and the learned distributed feature representation is mapped to the sample tag space. The output sample size of the second layer is set to 2 to determine whether it is an abusive comment, which plays the role of a binary classifier and calculates the probability of whether the final text is an abusive comment through softmax:

[0164] P = softmax(h cls W cls +b)

[0165] P label =P(C=1|T)

[0166] in, Belongs to learnable parameters, label num =2,h cls It represents the semantic representation of [CLS] after BERT encoding, T is the original text, and after linear transformation, the final prediction probability is obtained When label is 1, it means that the text is an abusive comment, and when label is 0, it means that the text is not an abusive comment.

[0167] It is understandable that when the updated classification model reaches convergence, an abusive comment detection model with the same structure can be constructed based on its parameters. When using the abusive comment detection model for comment detection, the comment text to be detected is first processed using a convolutional neural network with randomly initialized weights and a Chinese corpus to obtain corresponding word vectors and character vectors; the corresponding contextual semantic vectors are generated based on the word vectors and character vectors of the comment text to be detected; the contextual semantic vectors of the comment text to be detected are input into the abusive comment detection model to obtain the detection results of the comment text to be detected.

[0168] In one embodiment, if Fig. 9 As shown, step S100, obtaining the comment text to be detected, includes:

[0169] Step S101, obtaining a complete sentence of the comment to be detected;

[0170] Step S102, removing non-text characters in the complete sentence of the comment to be detected to obtain the comment text to be detected; wherein the non-text characters include emoticons, tags, and special characters.

[0171] It is understandable that in order to ensure the diversity and practicality of the data, the python crawler technology is used to crawl the real Internet comment text. After removing the web page tags, the data is preprocessed as follows:

[0172] First, we use a rule-based method to roughly filter the data, remove emoticons, tags, special characters, and unrecognizable characters, etc., to ensure that each comment is pure text data; since the length of the captured sample is not controlled, we need to set a maximum length and truncate each comment to finally form a sample data set D. Then we need to label the cleaned comment text, mainly using "crowdsourcing" technology to label and form a label.

[0173] From the above description, it can be seen that the abusive comment detection method provided by the present application forms the initial data set of the model through real Internet comment data collection, data preprocessing and labeling. Combined with the dual detection of strategy + algorithm, first, the establishment of the abusive vocabulary helps to ensure that the data containing obvious abusive semantics in the comments are directly detected, and the labor cost and later maintenance cost are reduced under the premise of ensuring accuracy; secondly, for comments that do not hit the abusive vocabulary, since there may be implicit abusive or abusive modal particles, in order to dig out the deep semantic information of the comments, a multi-level vector representation is selected, and the secondary pre-training model BERT whose parameters have been optimized in a large number of lexicons is selected to encode the comments. It can not only ensure that the encoder can fully represent the final semantic representation, but also further improve the accuracy and recall rate of the detection model. The present invention fully combines the advantages of strategy detection and algorithm detection, and through the clever combination of the two, it reduces labor costs, and maximizes the accuracy and effectiveness of model detection under the premise of minimum computing power.

[0174] From the software level, in order to solve the shortcomings of the existing abusive comment detection model, an embodiment of the abusive comment detection system provided in this application is shown in FIG. Fig.10 ,include:

[0175] The text acquisition module 1 is used to acquire the comment text to be detected;

[0176] A text filtering module 2, used for filtering the comment text to be detected;

[0177] The text detection module 3 is used to input the filtered comment text to be detected into a preset abusive comment detection model to obtain the detection result of the comment text to be detected; wherein, the abusive comment detection model is formed by training word vectors and character vectors corresponding to multiple historical comment texts.

[0178] In a specific implementation, the abusive comment detection system further includes a model training unit, which uses the historical comment texts with determined detection results to train the abusive comment detection model, the abusive comment detection model is a BERT model, and the BERT model includes a word segmentation layer (not shown in the figure), an embedded representation layer (including position embedding, segment embedding, token embedding), an encoding layer, and an output layer, see Fig.11 The word segmentation layer performs word segmentation on the speech text to obtain multiple single words and multiple words; the embedded representation layer encodes the speech text to obtain speech text encoding representation, word vector and word vector; the encoding layer encodes the speech text, word vector and word vector to obtain contextual semantic representation; the output layer obtains the detection result P of the speech text according to the contextual semantic representation tabel .

[0179] The abusive comment detection system also includes a rough detection unit, which can first detect the comment text to be detected, filter out some comment texts with obvious abusive meanings, and input the remaining comment texts with unclear abusive meanings into the abusive comment detection model for detection.

[0180] In a specific implementation, the model training unit and the coarse detection unit may be combined with a text detection module.

[0181] The specific process of the abusive comment detection system is as follows, see Fig.12 :

[0182] The data collection module group mainly crawls some text data sets about comments from some news and social platforms on the Internet, forms raw data and transmits it to the data preprocessing module.

[0183] The data preprocessing module mainly cleans dirty data according to certain rules, retains only comments in text form, and marks the data to form the initial data set required by the model.

[0184] The strategy rough detection module includes two parts: the establishment and judgment of the abuse vocabulary. Since the established abuse vocabulary is unambiguous and can affect the semantic information of the entire sentence, the content in the comment can be directly matched according to this table. Once a match is found, it is regarded as an abusive comment.

[0185] For the evaluation of the non-hit insult vocabulary, the model is trained. The training includes two parts: first, the pre-trained model BERT is pre-trained twice, so that it can learn the deep semantic information of this type of comment dataset and save the model parameters; second, the comments are represented by character-level vectors and word-level vectors using the Char-CNN model and google_zh_vocab in BERT, and then fused, and the contextual high-dimensional representation of the entire comment is obtained through the BERT model encoder.

[0186] Finally, through a two-layer feedforward neural network, the high-dimensional feature representation of [CLS] representing the entire sentence type information is reduced in dimension using linear transformation to achieve the purpose of binary classification. Then, the probability of calculating each comment is determined by the softmax function to determine whether it is an abusive comment.

[0187] The above is the specific process of training the entire model. Since the various parameters in the BERT model have been trained on a large number of data sets, it is only necessary to train the model for 3-5 epochs and save the training model of each epoch, select the optimal model using the validation set, and finally test the learned detection model with the test set to achieve the effect of detecting abusive comments.

[0188] From a hardware perspective, in order to address the shortcomings of existing detection models, the present application provides an embodiment of an electronic device for implementing all or part of the content of the abusive comment detection method, and the electronic device specifically includes the following content:

[0189] Fig.13 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig.13 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig.13 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0190] In one embodiment, the function of detecting abusive comments may be integrated into the central processing unit. The central processing unit may be configured to perform the following control:

[0191] Step S100: Obtain the comment text to be detected;

[0192] Step S200: filtering the comment text to be detected;

[0193] Step S300: inputting the filtered comment text to be detected into a preset abusive comment detection model to obtain a detection result of the comment text to be detected; wherein the abusive comment detection model is formed by training word vectors and character vectors corresponding to multiple historical comment texts.

[0194] The electronic device provided by the present invention selects a multi-level vector representation, and selects a secondary pre-trained model BERT whose parameters have been optimized in a large number of lexicons to encode the comments. It can not only ensure that the encoder can fully represent the final semantic representation, but also further improve the accuracy and recall rate of the detection model. The present invention fully combines the advantages of strategy detection and algorithm detection, and through the clever combination of the two, reduces labor costs, and maximizes the accuracy and effectiveness of model detection under the premise of minimum computing power.

[0195] In another embodiment, the abusive comment detection system may be configured separately from the central processor 9100. For example, the abusive comment detection system may be configured as a chip connected to the central processor 9100, and the abusive comment detection function may be implemented under the control of the central processor.

[0196] like Fig.13 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig.13 In addition, the electronic device 9600 may also include Fig.13 For components not shown, reference may be made to the prior art.

[0197] like Fig.13 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0198] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0199] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0200] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0201] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0202] The communication module 9110 is a transmitter / receiver 9110 that sends and receives signals via an antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0203] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the method for detecting abusive comments in the above embodiments. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, all the steps of the method for detecting abusive comments in the above embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0204] Step S100: Obtain the comment text to be detected;

[0205] Step S200: filtering the comment text to be detected;

[0206] Step S300: inputting the filtered comment text to be detected into a preset abusive comment detection model to obtain a detection result of the comment text to be detected; wherein the abusive comment detection model is formed by training word vectors and character vectors corresponding to multiple historical comment texts.

[0207] The computer-readable medium provided by the present application selects a multi-level vector representation, and selects the secondary pre-training model BERT whose parameters have been optimized in a large number of lexicons to encode the comments. It can not only ensure that the encoder can fully represent the final semantic representation, but also further improve the accuracy and recall rate of the detection model. The present invention fully combines the advantages of strategy detection and algorithm detection, and through the clever combination of the two, reduces labor costs, and maximizes the accuracy and effectiveness of model detection under the premise of minimum computing power.

[0208] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0209] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0210] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0211] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0212] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for detecting abusive comments, characterized in that: include: Get the comment text to be detected; The review text to be detected is truncated according to a preset maximum length to obtain multiple words to be detected; Performing preliminary detection on the plurality of words to be detected according to a preset abusive vocabulary list; If at least one of the multiple words to be detected matches an abusive word in the abusive vocabulary list, determining that the result of the preliminary detection is that the comment text to be detected is an abusive comment; Filtering the comment text to be detected; Input the filtered comment text to be detected into the preset abusive comment detection model to obtain the detection result of the comment text to be detected; The abusive comment detection model is formed by training word vectors and character vectors corresponding to multiple historical comment texts; The preliminary detection of the plurality of words to be detected according to the preset abusive vocabulary list includes: Performing a word embedding operation on each of the words to be detected and each of the abusive words respectively to obtain a first word vector corresponding to each of the words to be detected and a second word vector corresponding to each of the abusive words; For each first word vector, calculate the cosine value of the angle between it and each second word vector; When the cosine value of the angle exceeds a preset threshold, it is considered that the to-be-detected word corresponding to the first word vector matches the abusive word corresponding to the second word vector; The steps of training the abusive comment detection model include: Collecting a plurality of historical review texts, and filtering each historical review text according to the abusive vocabulary list to obtain training historical review texts that do not hit the abusive vocabulary list; The training historical review text is processed using a convolutional neural network with randomly initialized weights and a Chinese corpus to obtain corresponding word vectors and character vectors; Pre-train the preset classification model to enable it to learn the deep semantic information of the review text; The pre-trained classification model is trained again using the word vector and character vector to obtain the abusive comment detection model.

2. The method for detecting abusive comments according to claim 1, characterized in that: The filtering of the comment text to be detected is specifically as follows: Filter out the comment texts to be tested that are not abusive comments in the preliminary test results; The filtered comment text to be detected is input into a preset abusive comment detection model, specifically: The comment text to be detected, which is not an abusive comment as a result of preliminary detection, is input into a preset abusive comment detection model.

3. The method for detecting abusive comments according to claim 1, characterized in that: The convolutional neural network with randomly initialized weights and the Chinese corpus are used to process the training historical comment text to obtain corresponding word vectors and character vectors, including: Separating the training history review text into characters to obtain corresponding multiple words; Encoding the multiple words using a convolutional neural network with randomly initialized weights to obtain word vectors corresponding to the training historical comment text; and Performing word segmentation processing on the training history review text to obtain corresponding multiple words; The multiple words are represented according to a preset Chinese corpus to obtain word vectors corresponding to the training historical comment text.

4. The method for detecting abusive comments according to claim 1, characterized in that: The second training of the pre-trained classification model using the word vector and the character vector includes: Generate a corresponding context semantic vector according to the word vector and character vector corresponding to the training historical comment text; Inputting the context semantic vector corresponding to the training historical comment text and the preset true classification result into the pre-trained classification model to obtain an updated classification model; Repeat the above steps to train and update the classification model until the updated classification model converges; A neural network model is constructed according to the converged classification model parameters to obtain the abusive comment detection model.

5. The method for detecting abusive comments according to claim 4, characterized in that: The generating a corresponding context semantic vector according to the word vector and character vector corresponding to the training historical comment text includes: Add a beginning indicating word and an ending indicating word at the beginning and the end of each training history comment text, respectively, to obtain the input text corresponding to each training history comment text; The word vectors and character vectors corresponding to the training history comment texts and the input text are input into an encoder to obtain context semantic vectors of each training history comment text.

6. The method for detecting abusive comments according to claim 4, characterized in that: The pre-trained classification model includes two layers of feed-forward neural network linear layers; The context semantic vector corresponding to the training historical comment text and the preset real classification result are inputted into the pre-trained classification model to obtain an updated classification model, including: Inputting the context semantic vector corresponding to the training historical comment text and the preset classification result into the first-layer forward neural network linear layer for dimensionality reduction processing, so that the first-layer forward neural network linear layer outputs the dimensionality reduction feature vector of the context semantic vector; Inputting the dimension-reduced feature vector of the context semantic vector into the second-layer feed-forward neural network linear layer, so that the second-layer feed-forward neural network linear layer outputs the detection result of the training history comment text; The parameters of the classification model are updated according to the detection results and the corresponding true classification results to obtain an updated classification model.

7. The method for detecting abusive comments according to claim 6, characterized in that: The filtered comment text to be detected is input into a preset abusive comment detection model to obtain a detection result of the comment text to be detected, including: A convolutional neural network with randomly initialized weights and a Chinese corpus are used to process the review text to be detected, and the corresponding word vectors and character vectors are obtained; Generate the corresponding context semantic vector based on the word vector and character vector of the comment text to be detected; The context semantic vector of the comment text to be detected is input into the abusive comment detection model to obtain the detection result of the comment text to be detected.

8. The method for detecting abusive comments according to claim 1, characterized in that: The step of obtaining the comment text to be detected includes: Get the complete sentence of the comment to be tested; Removing non-text characters from the complete sentence of the comment to be detected to obtain the text of the comment to be detected; The non-text characters include emoticons, tags, and special characters.

9. A system for detecting abusive comments, characterized in that: include: A text acquisition module is used to obtain the comment text to be detected; a rough detection unit, configured to perform preliminary detection on a plurality of to-be-detected words according to a preset abusive vocabulary list, wherein the plurality of to-be-detected words are obtained by truncating the to-be-detected comment text according to a preset maximum length; if at least one of the plurality of to-be-detected words matches an abusive word in the abusive vocabulary list, then determining that the result of the preliminary detection is that the to-be-detected comment text is an abusive comment; A text filtering module, used for filtering the comment text to be detected; A text detection module, used to input the filtered comment text to be detected into a preset abusive comment detection model to obtain a detection result of the comment text to be detected; The abusive comment detection model is formed by training word vectors and character vectors corresponding to multiple historical comment texts; The preliminary detection of the plurality of words to be detected according to the preset abusive vocabulary list includes: Performing a word embedding operation on each of the words to be detected and each of the abusive words respectively to obtain a first word vector corresponding to each of the words to be detected and a second word vector corresponding to each of the abusive words; For each first word vector, calculate the cosine value of the angle between it and each second word vector; When the cosine value of the angle exceeds a preset threshold, it is considered that the to-be-detected word corresponding to the first word vector matches the abusive word corresponding to the second word vector; The steps of training the abusive comment detection model include: Collecting a plurality of historical review texts, and filtering each historical review text according to the abusive vocabulary list to obtain training historical review texts that do not hit the abusive vocabulary list; The training historical review text is processed using a convolutional neural network with randomly initialized weights and a Chinese corpus to obtain corresponding word vectors and character vectors; Pre-train the preset classification model to enable it to learn the deep semantic information of the review text; The pre-trained classification model is trained again using the word vector and character vector to obtain the abusive comment detection model.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for detecting abusive comments as described in any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting abusive comments as described in any one of claims 1 to 8 is implemented.

Images