A method, device and server for identifying insulting content

By extracting the semantic feature matrix and weight vector of the comment text, combining the probability of insult and negative emotions, we can identify whether the comment text is abusive content, which solves the problem of low accuracy in the recognition of insult content in the existing technology, and improves the recognition of insult content accuracy.

CN111930941BActive Publication Date: 2024-11-19TENCENT MUSIC ENTERTAINMENT TECH (SHENZHEN) CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202010761006.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-07-31
Publication Date
2024-11-19
Estimated Expiration
2040-07-31

AI Technical Summary

Technical Problem

The existing abusive content recognition methods have misjudgment problems in social scenarios, especially the multiple expressions and inconsistent meanings of the same abusive keyword lead to low accuracy in the recognition of abusive content.

Method used

By judging whether the comment words in the comment text contain keywords in the abusive text library, extract the semantic feature matrix, calculate the abusive probability and negative emotion probability, combine the first and second weight vectors to determine whether the comment text is abusive content, and generate a report information.

Benefits of technology

It improves the accuracy of recognition of abusive content, avoids misjudgment of texts with positive emotions, and enhances the accuracy of recognition of abusive content.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111930941B_ABST
    Figure CN111930941B_ABST
Patent Text Reader

Abstract

The embodiment of the present application discloses a method and device for identifying insulting content, and a server, including: determining whether multiple comment words in a comment text contain any insulting keyword in an insulting text library; if the comment text contains any insulting keyword, extracting the semantic feature matrix of the comment text; calculating the insulting probability of the comment text according to the semantic feature matrix of the comment text; obtaining the negative emotion probability of the comment text according to the semantic feature matrix of the comment text and a first weight vector; determining whether the comment text is an insulting comment text according to the insulting probability and the negative emotion probability. The present application can improve the recognition accuracy of insulting content.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for identifying insulting content, and a server. Background Art

[0002] At present, the solution for identifying abusive content mainly uses text classification models to identify abusive content. Among them, the commonly used algorithms of text classification models include fasttext, textcnn, lstm and other related algorithms. However, in common social scenarios, the same abusive keyword, such as pig, has multiple expressions, and the insulting meanings expressed are also inconsistent. For example, silly pig is so cute. This expression does not have an insulting meaning, but is an intimate expression. If a text classification model is used to identify abusive content, the abusive content may be misjudged, resulting in a low accuracy rate in identifying the abusive content.

[0003] Application Contents

[0004] The embodiments of the present application provide a method and device for identifying insulting content, a server, and a storage medium, in order to improve the accuracy of identifying insulting content.

[0005] On the one hand, an embodiment of the present application provides a method for identifying insulting content, including:

[0006] Determine whether multiple comment words in the comment text contain any insulting keyword in the insulting text library;

[0007] If the comment text contains any of the insulting keywords, extracting a semantic feature matrix of the comment text;

[0008] Calculating the insult probability of the comment text according to the semantic feature matrix of the comment text;

[0009] Obtaining the negative sentiment probability of the comment text according to the semantic feature matrix of the comment text and the first weight vector, wherein the first weight vector is obtained by training based on the first sample comment set and the sentiment classification label corresponding to each comment text in the first sample comment set, and the first weight vector includes the sentiment classification weight of each comment word in the multiple comment words;

[0010] According to the insult probability and the negative emotion probability, it is determined whether the comment text is an insult comment text.

[0011] Optionally, obtaining the negative sentiment probability of the comment text according to the semantic feature matrix and the first weight vector of the comment text includes:

[0012] Obtaining a first text feature vector of the comment text according to the semantic feature matrix of the comment text and the first weight vector;

[0013] The first text feature vector is converted into a probability sequence to obtain the negative sentiment probability of the comment text.

[0014] Optionally, the extracting semantic features of the comment text includes:

[0015] According to the order of appearance of each comment word in the comment text, the initial word vector of each comment word is sorted to obtain the initial matrix of the comment text;

[0016] Obtaining a text matrix of the comment text according to the initial matrix of the comment text and the input weight matrix, wherein the input weight matrix includes pre-trained input weight column vectors of a preset length;

[0017] Feature extraction is performed on the text matrix to obtain a semantic feature matrix of the comment text.

[0018] Optionally, before obtaining the text matrix of the comment text according to the initial matrix of the comment text and the input weight matrix, the method further comprises:

[0019] The initial word vectors of each comment word in each comment text in the second sample comment set are sorted to obtain the initial matrix of each comment text, and then the initial matrix of the second sample comment set is obtained, wherein the second sample comment set includes comment text a i , the comment text a i Include target review word b j , where i is a positive integer less than or equal to the total number of comment texts corresponding to the second sample comment set, and j is a positive integer less than or equal to the number of comment texts a i A positive integer corresponding to the total number of comment words;

[0020] The comment text a is obtained according to the initial matrix, the initial input weight matrix and the initial output weight matrix of the second sample comment set. i Target review word b j The corresponding output vector is used to obtain the target comment word b according to the output vector j The predicted word vector of

[0021] According to the target review word b j The predicted word vector and the target review word b j The initial word vector of , adjusting the initial input weight matrix and the output weight matrix;

[0022] When the adjusted target review word b j The predicted word vector and the target review word b j When the initial word vector satisfies the first convergence condition, the adjusted initial input weight matrix is ​​determined as the input weight matrix.

[0023] Optionally, the calculating the insult probability of the comment text according to the semantic feature matrix of the comment text includes:

[0024] Obtaining a second text feature vector according to the semantic feature matrix of the comment text and the second weight vector, wherein the second weight vector includes the insult recognition weight of each comment word in the plurality of comment words, and the weight of the insult comment words in the second weight vector is higher than the weight of the non-insult comment words;

[0025] The second text feature vector is converted into a probability sequence to obtain the insult probability of the comment text.

[0026] Optionally, before obtaining the first text feature vector of the comment text according to the semantic feature matrix and the first weight vector of the comment text, the method further includes:

[0027] Extracting a semantic feature matrix of each comment text in the third sample comment set to obtain a semantic feature matrix of the third sample comment set;

[0028] Obtaining a text feature vector of the third sample comment set according to the semantic feature matrix of the third sample comment set and the first initial weight vector;

[0029] Converting the text feature vector of the third sample comment set into a probability sequence to obtain a predicted sentiment classification label of each comment text in the third sample comment set;

[0030] Adjusting the first initial weight vector according to the predicted sentiment classification label and the sentiment classification label of each comment text in the third sample comment set;

[0031] When the predicted sentiment classification labels and sentiment classification labels of the comment texts in the third sample comment set meet the second convergence condition, the adjusted first initial weight vector is determined as the first weight vector.

[0032] Optionally, the text feature vector of the third sample comment set includes the mth element X m , m is a positive integer less than or equal to the total number M of comment texts corresponding to the third sample comment set;

[0033] The converting the text feature vector of the third sample comment set into a probability sequence comprises:

[0034] Calculate the mth element X m The probability value Wherein, n is a positive integer greater than or equal to 1 and less than or equal to M;

[0035] The probability value of each element in the text feature vector of the third sample comment set is obtained according to the probability value Sm, and the probability sequence corresponding to the text feature vector of the third sample comment set is obtained according to the probability value of each element.

[0036] Optionally, the insulting comment text carries a user account corresponding to the insulting comment text;

[0037] After determining that the comment text is an insulting comment text, the method further includes:

[0038] A report message is generated according to the insulting comment text and the user account, and the report message is sent to a network management platform.

[0039] In one aspect, an embodiment of the present application provides a device for identifying insulting content, including:

[0040] A judgment module, used for judging whether a plurality of comment words in the comment text contain any insulting keyword in the insulting text library;

[0041] determining an extraction module, for extracting a semantic feature matrix of the comment text if the comment text contains any of the insulting keywords;

[0042] An insult probability calculation module, used to calculate the insult probability corresponding to the comment text according to the semantic feature matrix of the comment text;

[0043] A negative emotion probability calculation module, used to obtain the negative emotion probability of the comment text according to the semantic feature matrix of the comment text and a first weight vector, wherein the first weight vector is trained based on the first sample comment set and the emotion classification label corresponding to each comment text in the first sample comment set, and the first weight vector includes the emotion classification weight of each comment word in the multiple comment words;

[0044] The determination module is used to determine whether the comment text is an insulting comment text according to the insult probability and the negative emotion probability.

[0045] Optionally, the negative emotion probability calculation module is specifically used to:

[0046] Obtaining a first text feature vector of the comment text according to the semantic feature matrix of the comment text and the first weight vector;

[0047] The first text feature vector is converted into a probability sequence to obtain the negative sentiment probability of the comment text.

[0048] Optionally, the determining the extraction module includes:

[0049] A first initial matrix determination unit, used to sort the initial word vectors of each comment word in the comment text according to the order of appearance of each comment word in the comment text, so as to obtain an initial matrix of the comment text;

[0050] A text matrix calculation unit, configured to obtain a text matrix of the comment text according to an initial matrix of the comment text and an input weight matrix, wherein the input weight matrix includes a pre-trained input weight column vector of a preset length;

[0051] The feature extraction unit is used to extract features from the text matrix to obtain a semantic feature matrix of the comment text.

[0052] Optionally, the determining and extracting module further includes:

[0053] The second initial matrix determination unit is used to sort the initial word vectors of each comment word in each comment text in the second sample comment set to obtain the initial matrix of each comment text, and then obtain the initial matrix of the second sample comment set, wherein the second sample comment set includes comment text a i , the comment text a i Include target review word b j , where i is a positive integer less than or equal to the total number of comment texts corresponding to the second sample comment set, and j is a positive integer less than or equal to the number of comment texts a i A positive integer corresponding to the total number of comment words;

[0054] A calculation unit is used to obtain the comment text a according to the initial matrix, the initial input weight matrix and the initial output weight matrix of the second sample comment set. i Target review word b j The corresponding output vector is obtained according to the output vector. j The predicted word vector of

[0055] Adjust the weight matrix unit to adjust the target review word b j The predicted word vector and the target review word b jThe initial word vector of the input weight matrix and the output weight matrix are adjusted;

[0056] The input weight matrix determination unit is used to adjust the target comment word b j The predicted word vector and the target review word b j When the initial word vector satisfies the first convergence condition, the adjusted initial input weight matrix is ​​determined as the input weight matrix.

[0057] Optionally, the module for calculating the probability of verbal abuse includes:

[0058] A second text feature calculation unit is used to obtain a second text feature vector according to the semantic feature matrix of the comment text and the second weight vector, wherein the second weight vector includes the insult recognition weight of each comment word in the plurality of comment words, and the weight of the insult comment words in the second weight vector is higher than the weight of the non-insult comment words;

[0059] The second probability conversion unit is used to convert the second text feature vector into a probability sequence to obtain the insult probability of the comment text.

[0060] Optionally, the device further includes: a weight vector determination module.

[0061] The weight vector determination module comprises:

[0062] A semantic feature extraction unit is used to extract a semantic feature matrix of each comment text in the third sample comment set to obtain a semantic feature matrix of the third sample comment set;

[0063] A text feature calculation unit, configured to obtain a text feature vector of the third sample comment set according to the semantic feature matrix of the third sample comment set and the first initial weight vector;

[0064] A predicted emotion label determination unit, used to convert the text feature vector of the third sample comment set into a probability sequence to obtain a predicted emotion classification label of each comment text in the third sample comment set;

[0065] an adjusting unit, configured to adjust the first initial weight vector according to the predicted sentiment classification label and the sentiment classification label of each comment text in the third sample comment set;

[0066] The first weight vector determining unit is used to determine the adjusted first initial weight vector as the first weight vector when the predicted sentiment classification label and the sentiment classification label of the comment text in the third sample comment set meet the second convergence condition.

[0067] Optionally, the text feature vector of the third sample comment set includes the mth element X m , m is a positive integer less than or equal to the total number M of comment texts corresponding to the third sample comment set;

[0068] The predicted emotion classification label determination unit comprises:

[0069] The probability value calculation subunit is used to calculate the mth element X m The probability value Wherein, n is a positive integer greater than or equal to 1 and less than or equal to M;

[0070] The probability sequence determination subunit is used to obtain the probability value of each element in the text feature vector of the third sample comment set according to the probability value Sm, and obtain the probability sequence corresponding to the text feature vector of the third sample comment set according to the probability value of each element.

[0071] Optionally, the insulting comment text carries a user account corresponding to the insulting comment text;

[0072] The device also includes:

[0073] A generating and sending module is used to generate a reporting message according to the insulting comment text and the user account, and send the reporting message to a network management platform.

[0074] On the one hand, an embodiment of the present application provides a server, including a processor, a memory and a transceiver, wherein the processor, the memory and the transceiver are interconnected, wherein the memory is used to store a computer program that supports the abusive content identification device to execute the above-mentioned abusive content identification method, and the computer program includes program instructions; the processor is configured to call the program instructions to execute the abusive content identification method as described in the above-mentioned first embodiment of the present application.

[0075] In one aspect, an embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, wherein the computer program includes program instructions; when the program instructions are executed by a processor, the processor executes the method for identifying insulting content as described above in one aspect of the embodiment of the present application.

[0076] In an embodiment of the present application, the insult content recognition platform determines whether multiple comment words in a comment text contain any insult keyword in an insult text library; if the comment text contains any insult keyword, the semantic feature matrix of the comment text is extracted; the insult probability of the comment text is calculated based on the semantic feature matrix of the comment text; the negative emotion probability of the comment text is obtained based on the semantic feature matrix of the comment text and the first weight vector; and whether the comment text is an insult comment text is determined based on the insult probability and the negative emotion probability. The present application can improve the recognition accuracy of insult content. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0078] Figure 1 It is a schematic diagram of a system architecture provided by an embodiment of the present application;

[0079] Figure 2 It is a flowchart of a method for identifying insulting content provided by an embodiment of the present application;

[0080] Figure 3 It is a schematic diagram of a network structure based on word2vec+bi-lstm+attention algorithm provided in an embodiment of the present application;

[0081] Figure 4 It is a flowchart of a method for identifying insulting content provided by an embodiment of the present application;

[0082] Figure 5 is a structural diagram of an insult content identification device provided in an embodiment of the present application;

[0083] Figure 6 It is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0084] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0085] See also Figure 1, is a schematic diagram of a system architecture provided by an embodiment of the present application. Figure 1 As shown in FIG. 1 , the system architecture diagram includes the comment publisher and its terminal, the insult content identification platform and the network management platform, wherein the comment publisher and its terminal, such as Figure 1 As shown, it may specifically include comment posting party a and its terminal, comment posting party b and its terminal, ...

[0086] Among them, the abusive content identification platform, the terminal where each comment poster is located, and the network management platform can be computer devices, including mobile phones, tablets, laptops, PDAs, smart speakers, mobile Internet devices (MID, mobile internet device), POS (Point Of Sales, point of sales) machines, wearable devices (such as smart watches, smart bracelets, etc.), etc.

[0087] Furthermore, if Figure 1 As shown, in the process of implementing the method for identifying insulting content, the comment posting party a posts a comment text on its terminal, such as Figure 1 As shown in the human-computer interaction interface 100 in the figure, the insult content identification platform obtains the comment text, determines whether multiple comment words in the comment text contain any insulting keyword in the insulting text library, and if the comment text contains any insulting keyword in the insulting text library, extracts the semantic feature matrix of the comment text, and calculates the insult probability of the target comment text according to the semantic feature matrix of the comment text. If the insult probability is greater than a first preset threshold, obtains a first text feature vector of the comment text according to the semantic feature matrix of the comment text and a first weight vector, converts the first text feature vector into a probability sequence, and obtains the negative emotion probability of the comment text. If the negative emotion probability is greater than a second preset threshold, the comment text is determined to be an insulting comment text, and generates a report message according to the insulting comment text and the user account carried by the insulting comment text, and sends the report message to the network management platform, and the network management platform performs corresponding crackdown on the user account.

[0088] For further information, see Figure 2 , is a flow chart of a method for identifying insulting content provided by an embodiment of the present application. Figure 2 As shown, the method embodiment includes the following steps:

[0089] S101, determining whether a plurality of comment words in a comment text contain any insulting keyword in an insulting text library.

[0090] Among them, the insulting keywords are the same characters in various words with insulting meanings.

[0091] Specifically, the insulting content identification platform compares the kth word in the comment text with the insulting keywords in the insulting text library, where k is a positive integer less than or equal to the total number of words in the comment text. If the kth word in the comment text is consistent with any insulting keyword in the insulting text library, it is determined that the comment text contains the insulting keywords in the insulting text library, and then step S102 is executed.

[0092] S102: If the comment text contains any insulting keyword, extract the semantic feature matrix of the comment text.

[0093] Optionally, the semantic feature matrix of the extracted comment text includes:

[0094] According to the order of appearance of each comment word in the comment text, the initial word vector of each comment word is sorted to obtain the initial matrix of the comment text;

[0095] Obtaining a text matrix of the comment text according to the initial matrix of the comment text and the input weight matrix, wherein the input weight matrix includes a preset value of pre-trained input weight column vectors;

[0096] Feature extraction is performed on the text matrix to obtain a semantic feature matrix of the comment text.

[0097] Specifically, after determining that the comment text contains any abusive keyword, the abusive content identification platform converts each comment word in the comment text into a discrete symbol through a One-Hot Encoder. That is, each comment word in the comment text corresponds to a row vector, in which only one value is 1 and the rest are 0. That is, the row vector corresponding to each comment word is the initial word vector of the comment word, where the dimension of the initial word vector is set manually and is not limited here.

[0098] Afterwards, since the number of comment words contained in different comment texts is different, in order to make the number of rows and columns of the initial matrix of each comment text consistent, the initial word vector of each comment word in the comment text is sorted according to the order of appearance of each comment word in the comment text to obtain a matrix of m rows and n columns. Afterwards, the number of comment words l contained in the comment text is compared with the preset number of rows L of the initial matrix. If the number of comment words is less than the preset number of rows of the initial matrix, then after adding (Ll) n-dimensional zero vectors in the m+1th row of the above-mentioned matrix of m rows and n columns, a matrix of L rows and n columns is obtained, that is, the initial matrix of the comment text. For example, the initial word vectors of the comment words "I", "love", "motherland" and "mother" in the comment text "I love my motherland" are x1, x2, x3 and x4 respectively, then the initial matrix of the comment text is (x1, x2, x3, x4, x5, x6, ...) T , where the vectors “x5, x6, …” are all zero vectors with the same dimension as the initial word vectors “x1, x2, x3 and x4”.

[0099] Understandable, see Figure 3 , is a schematic diagram of a network structure based on word2vec+bi-lstm+attention algorithm provided in an embodiment of the present application. Here, the initial matrix of the comment text, i.e. Figure 3 The network structure input in the input layer (x1, x2, x3, ..., x T ) T , where x1,x2,x3,…,x T The initial word vectors of multiple comment words in the comment text are included, and the number of rows T of the network structure input is an integer greater than or equal to the total number of comment words contained in the comment text. In other words, the input layer in the network structure is used to convert each comment word in the comment text into an initial word vector.

[0100] Furthermore, the text matrix of the comment text is obtained according to the initial matrix of the comment text and the input weight matrix. Since the input weight matrix is ​​pre-trained, the input weight matrix needs to be trained before the text matrix of the comment text is calculated.

[0101] Optionally, before obtaining the text matrix of the text according to the initial matrix of the comment text and the input weight matrix, the method further comprises:

[0102] The initial word vectors of each comment word in each comment text in the second sample comment set are sorted to obtain the initial matrix of each comment text, and then the initial matrix of the second sample comment set is obtained, wherein the second sample comment set includes comment text ai , the comment text a i Include target review word b j , where i is a positive integer less than or equal to the total number of comment texts corresponding to the second sample comment set, and j is a positive integer less than or equal to the number of comment texts a i A positive integer corresponding to the total number of comment words;

[0103] The comment text a is obtained according to the initial matrix, the initial input weight matrix and the initial output weight matrix of the second sample comment set. i Target review word b j The corresponding output vector is used to obtain the target comment word b according to the output vector j The predicted word vector of

[0104] According to the target review word b j The predicted word vector and the target review word b j The initial word vector of , adjusting the initial input weight matrix and the output weight matrix;

[0105] When the adjusted target review word b j The predicted word vector and the target review word b j When the initial word vector satisfies the first convergence condition, the adjusted initial input weight matrix is ​​determined as the input weight matrix.

[0106] Among them, the number of rows of the input weight matrix is ​​the dimension of the initial word vector of each comment word, and the number of columns is the number of neurons in the hidden layer. The input weight matrix is ​​used to map the dimension of the initial word vector of each comment word into a vector of fixed dimension, thereby avoiding the dimensionality explosion of the initial word vector of each comment word. In addition, the number of rows and columns of the initial input weight matrix is ​​consistent with the number of rows and columns of the input weight matrix, and the number of rows and columns of the initial output weight matrix are consistent with the number of columns and rows of the initial input weight matrix, respectively.

[0107] Specifically, each comment text a in the second sample comment set is obtained by obtaining the initial matrix of the comment text according to the above comment text. i (excluding target review word b j )’s initial matrix X i , i is a positive integer less than or equal to the total number of comment texts I included in the second sample comment set, j is a positive integer less than or equal to the number of comment texts a i The total number of comment words contained in is a positive integer, thereby obtaining the initial matrix X of the second comment sample set = (X1, X2, ..., X I ) T, multiply the initial matrix X of the second comment sample set by the initial input weight matrix W1, add the values ​​of each column in the matrix to get the average, and get the hidden layer vector with the same dimension as the number of columns (number of hidden layer neurons) of the initial input weight matrix, and multiply the hidden layer vector by the initial output weight matrix to get the comment text a i Target review word b j The corresponding output vector is then converted into the target comment word b through the activation function softmax. j Each element in the corresponding output vector is mapped to a real number between 0 and 1, and the target comment word b is obtained. j The predicted word vector of the comment text a i Target review word b j The predicted word vector and comment text a i Target review word b j The initial word vector of the initial input weight matrix and the initial output weight matrix are adjusted. When the adjusted comment text a i Target review word b j The predicted word vector and comment text a i Target review word b j If the initial word vector of satisfies the first convergence condition, the adjusted initial input weight matrix is ​​determined as the input weight matrix. For example, the first convergence condition is that the adjusted comment text a i Target review word b j The predicted word vector of each element value and comment text a i Target review word b j The absolute value of the error between each element value in the initial word vector is minimized.

[0108] It can be understood that the process of training the input weight matrix according to the initial word vector of each comment word in the second comment sample set and each comment text in the second sample comment set in the embodiment of the present application is actually the continuous bag of words (CBOW) model in the Word2vec model, that is, a neural network model with 1 hidden layer. The word vector of the target comment word is predicted by the initial word vectors of other comment words except the target comment word in each comment text to obtain the predicted word vector of the target comment word. By adjusting the initial input weight matrix and the initial output weight matrix in the CBOW model, the error between each element value in the predicted word vector of the target comment word and each element value in the initial word vector of the target comment word is continuously reduced. When the error value between each element is minimized, the adjusted initial input weight matrix is ​​determined as the input weight matrix.

[0109] After that, the initial matrix of the comment text is multiplied by the input weight matrix containing the preset value (the number of neurons in the hidden layer) of the input weight column vector to obtain the text matrix of the comment text. It can be understood that the implementation process of the abusive content identification platform to obtain the text matrix of the comment text based on the initial matrix of the comment text and the input weight matrix can be achieved through Figure 3 The embedding layer in the network structure shown, namely, the Embedding Layer implementation, is used to map the dimension of the initial word vector of each comment word in the comment text into a vector with a preset dimension.

[0110] Afterwards, the text matrix is ​​feature extracted through the bidirectional long short-term memory network (Bi-LSTM) model, and the forward output and reverse output of the i-th comment word are obtained as hi' and hi respectively. The corresponding elements in hi' and hi are added to obtain the semantic feature vector of the i-th comment word, and then the semantic feature matrix of the comment text is obtained.

[0111] It is understandable that the implementation process of the insult content identification platform extracting the semantic feature matrix of the comment text based on the text matrix of the comment text can be achieved through Figure 3 The bidirectional long short-term memory network layer, namely the Bi-lstm Layer, in the network structure shown in the figure extracts the context features of each comment word through the Bi-LSTM model, obtains the semantic feature vector of each comment word according to the context features of each comment word, and thus obtains the semantic feature matrix of the comment text.

[0112] S103, calculating the insult probability of the comment text according to the semantic feature matrix of the comment text.

[0113] Specifically, the semantic feature matrix of the comment text is input into the insult tendency recognition model to obtain the insult probability of the target comment text.

[0114] The insult tendency recognition model is trained based on the third sample comment set and the insult classification label corresponding to each comment text in the third sample comment set.

[0115] Optionally, inputting the semantic features of the comment text into an insult tendency recognition model to obtain an insult probability of the comment text includes:

[0116] Obtaining a second text feature vector according to the semantic feature matrix of the comment text and the second weight vector, wherein the second weight vector includes the insult recognition weight of each comment word in the plurality of comment words, and the weight of the insult comment words in the second weight vector is higher than the weight of the non-insult comment words;

[0117] The second text feature vector is converted into a probability sequence to obtain the insult probability of the comment text.

[0118] The second weight vector is a column vector having the same number of rows as the number of rows of the initial matrix of the comment text in step S102, and the second weight vector is a model parameter of the insult tendency recognition model.

[0119] Specifically, the abusive content identification platform multiplies the transposed vector of the second weight vector by the semantic feature matrix of the comment text to obtain a second text feature vector, and maps the elements in the second text feature vector to real numbers between 0 and 1 through the activation function softmax to obtain the probability of abuse corresponding to the comment text.

[0120] Understandably, the model for identifying insult tendency here, i.e. Figure 3 The attention layer in the network structure shown, i.e., Attention Layer, and the output layer, wherein the attention layer is used to adjust the insult recognition weight of each comment word in the comment text, and the implementation method of adjusting the insult recognition weight of each comment word can be to increase the weight of the comment words with insulting meanings and reduce the weight of irrelevant words, and finally obtain the second weight vector. Here, the implementation process of obtaining the second weight vector can refer to the description of obtaining the first weight vector in step S104, which will not be repeated here. Afterwards, the attention layer performs weighted summation of the semantic feature vectors of each comment word in the comment text according to the second weight vector to obtain the overall features of the comment text, i.e., the second text feature vector of the comment text. The output layer is used to map the elements in the second text feature vector to real numbers between 0 and 1, i.e., the probability of insult corresponding to the comment text.

[0121] S104, obtaining the negative sentiment probability of the comment text according to the semantic feature matrix of the comment text and the first weight vector.

[0122] Specifically, the abusive content identification platform obtains a first text feature vector of the comment text according to the semantic feature matrix and the first weight vector of the comment text; and converts the first text feature vector into a probability sequence to obtain the negative emotion probability of the comment text.

[0123] The abusive content identification platform obtains a first text feature vector of the comment text according to the semantic feature matrix and the first weight vector of the comment text.

[0124] The first weight vector is trained based on the first sample comment set and the sentiment classification label corresponding to each comment text in the first sample comment set, and the first weight vector includes the sentiment classification weight of each comment word in the multiple comment words. In addition, the sentiment classification label corresponding to each comment text is a positive sentiment label or a negative sentiment label.

[0125] Before executing step S104, the insult content identification platform obtains a first weight vector through training based on the second sample comment set and the sentiment classification label corresponding to each comment text in the second sample comment set.

[0126] Optionally, before obtaining the first text feature vector of the comment text according to the semantic feature matrix and the first weight vector of the comment text, the method further includes:

[0127] Extracting a semantic feature matrix of each comment text in the third sample comment set to obtain a semantic feature matrix of the third sample comment set;

[0128] Obtaining a text feature vector of the third sample comment set according to the semantic feature matrix of the third sample comment set and the first initial weight vector;

[0129] Converting the text feature vector of the third sample comment set into a probability sequence to obtain a predicted sentiment classification label of each comment text in the third sample comment set;

[0130] Adjusting the first initial weight vector according to the predicted sentiment classification label and the sentiment classification label of each comment text in the third sample comment set;

[0131] When the predicted sentiment classification labels and sentiment classification labels of the comment texts in the third sample comment set meet the second convergence condition, the adjusted first initial weight vector is determined as the first weight vector.

[0132] Specifically, according to the method of extracting the semantic feature matrix of the comment text in step S102, the semantic feature matrix of each comment text in the third sample comment set is obtained, and then the semantic feature matrix of the third sample comment set is obtained, and the first initial weight vector and the semantic feature matrix of the third sample comment set are multiplied to obtain the text feature vector of the third sample comment set.

[0133] Furthermore, the abusive content identification platform converts the text feature vector of the third sample comment set into a probability sequence.

[0134] Optionally, the text feature vector of the third sample comment set includes the mth element X m, m is a positive integer less than or equal to the total number M of comment texts corresponding to the third sample comment set;

[0135] The converting the text feature vector of the third sample comment set into a probability sequence comprises:

[0136] The mth element X m The probability value Wherein, n is a positive integer greater than or equal to 1 and less than or equal to M;

[0137] The probability value of each element in the text feature vector of the third sample comment set is obtained according to the probability value Sm, and the probability sequence corresponding to the text feature vector of the third sample comment set is obtained according to the probability value of each element.

[0138] The number of elements contained in the text feature vector of the third sample comment set is the total number of comment texts contained in the third sample comment set, and the mth element X in the text feature vector is m is the text feature corresponding to the mth comment text in the third sample comment set. In addition, the sum of the probability values ​​of all elements in the text feature vector is 1.

[0139] For example, the text feature vector of the third sample comment set is (3, -1, 2, 5), then the probability value of the first element 3 in the text feature vector is S1 = e^3 / (e^3+e^(-1)+e^2+e^5). According to this method, the probability values ​​of the second, third and fourth elements in the text feature vector can be calculated as S2, S3 and S4 respectively. Then the probability sequence corresponding to the text feature vector of the third sample comment set is S1, S2, S3 and S4, that is, the negative sentiment probability of each comment text in the third sample comment set is obtained.

[0140] Exemplarily, if the negative sentiment probability of the i-th comment text is greater than or equal to a preset threshold, such as 0.6, the predicted sentiment classification label of the i-th comment text is determined as a negative sentiment label; otherwise, the predicted sentiment classification label of the i-th comment text is determined as a positive sentiment label.

[0141] Afterwards, the first initial weight vector is adjusted according to the predicted sentiment classification label and the sentiment classification label of each comment text in the third sample comment set. When the adjusted first initial weight vector meets the second convergence condition, the adjusted first initial weight vector is determined as the first weight vector. Exemplarily, the second convergence condition is that the ratio between the number of comment texts in the third sample comment set that meet the predicted sentiment classification label and the sentiment classification label consistent with the total number of comment texts included in the third sample comment set is greater than or equal to a preset ratio.

[0142] Furthermore, the abusive content identification platform multiplies the transposed vector of the first weight vector by the semantic feature matrix of the comment text to obtain a first text feature vector of the comment text.

[0143] It can be understood that the implementation process of obtaining the first text feature vector of the comment text according to the semantic feature matrix of the comment text and the first weight vector can be achieved by Figure 3 Implementation of the attention layer in the shown network structure.

[0144] Afterwards, the abusive content identification platform converts the first text feature vector into a probability sequence to obtain the negative sentiment probability of the comment text.

[0145] Specifically, the elements in the first text feature vector are mapped to real numbers between 0 and 1 through the activation function softmax to obtain the negative sentiment probability of the comment text.

[0146] It is understandable that the implementation process of the insult content identification platform converting the first text feature vector into a probability sequence can be achieved by Figure 3 Output layer implementation in the shown network structure.

[0147] S105, determining whether the comment text is an insulting comment text according to the insult probability and the negative emotion probability.

[0148] In a possible implementation, if the insult probability is greater than a first preset threshold, and the negative emotion probability is greater than a second preset threshold, the comment text is determined to be an insult comment text.

[0149] For example, if the insult probability of 0.7 and the negative emotion probability of 0.6 of the comment text are respectively greater than the first preset threshold of 0.6 and the second preset threshold of 0.5, then the target comment text is determined to be an insulting comment text.

[0150] In another possible implementation, the insult probability and the negative emotion probability are weightedly calculated according to the preset weight coefficients corresponding to each other to obtain a first probability. If the first probability is greater than a preset threshold, the comment text is determined to be an insulting comment text.

[0151] For example, according to the preset weight coefficients corresponding to the insult probability and the negative emotion probability, i.e., 0.4 and 0.6, the insult probability 0.7 and the negative emotion probability 0.6 of the comment text are weightedly calculated, and the first probability is 0.4*0.7+0.6*0.6=0.64, which is greater than the preset threshold value 0.5, and the comment text is determined to be an abusive comment text.

[0152] In the embodiment of the present application, when the abusive content identification platform identifies whether a comment text is an abusive comment text, in addition to judging whether the comment text has an abusive meaning based on the insult probability of the comment text, it also calculates the negative emotion probability of the comment text, and judges whether the comment text has negative emotion based on the negative emotion probability of the comment text. If the insult probability is greater than a first preset threshold, and the negative emotion probability is greater than a second preset threshold, the comment text is determined to be an abusive comment text. Therefore, the use of the present application can effectively avoid the misjudgment of text content with positive emotions and improve the recognition accuracy of abusive content.

[0153] See also Figure 4 , is a flow chart of a method for identifying insulting content provided by an embodiment of the present application. Figure 4 As shown, the method embodiment includes the following steps:

[0154] S201, determining whether a plurality of comment words in a comment text contain any insulting keyword in an insulting text library.

[0155] Specifically, the insult content identification platform compares the kth word in the comment text with the insult keywords in the insult text library, where k is a positive integer less than or equal to the total number of words in the comment text. If the kth word in the comment text is consistent with any insult keyword in the insult text library, it is determined that the comment text contains the insult keyword in the insult text library, and step S202 is executed. In addition, if every word in the comment text is inconsistent with any insult keyword in the insult text library, it is determined that the comment text does not contain any insult keyword in the insult text library, and step S210 is executed.

[0156] S202, extracting the semantic feature matrix of the comment text.

[0157] S203, calculating the insult probability of the comment text according to the semantic feature matrix of the comment text.

[0158] Here, the specific implementation of steps S202-S203 can be referred to Figure 2 The description of steps S102-S103 in the corresponding embodiment will not be repeated here.

[0159] S204, determining whether the probability of insult is greater than a first preset threshold.

[0160] Specifically, if the insult probability is greater than the first preset probability, step S205 is executed; otherwise, step S210 is executed.

[0161] S205, obtaining the negative sentiment probability of the comment text according to the semantic feature matrix of the comment text and the first weight vector.

[0162] Here, the specific implementation of step S205 can refer to Figure 2 The description of step S104 in the corresponding embodiment will not be repeated here.

[0163] S206, determining whether the probability of negative emotion is greater than a second preset threshold.

[0164] Specifically, if the negative emotion probability is greater than the second preset threshold, execute step S207, otherwise, execute step S209.

[0165] S207, determining that the comment text is an insulting comment text, and the insulting comment text carries the user account corresponding to the insulting comment text.

[0166] S208, generating a report message according to the insulting comment text and the user account, and sending the report message to the network management platform.

[0167] Specifically, the abusive content identification platform generates a report message based on the abusive comment text and the user account, and sends the report message to the network management platform. After receiving the report message, the network management platform bans the user account for a preset period of time or cancels the user account.

[0168] S209, post comment text.

[0169] In an embodiment of the present application, the insult content recognition platform determines whether the comment text contains any insult keyword. If it does not contain any insult keyword, the comment text is published; if it contains any insult keyword, the semantic feature matrix of the comment text is extracted, and the semantic feature matrix is ​​input into the insult tendency recognition model to obtain the insult probability corresponding to the comment text. If the insult probability is less than or equal to the first preset threshold, the comment text is published; if the insult probability is greater than the first preset threshold, the first text feature vector of the comment text is obtained according to the semantic feature matrix of the comment text and the first weight vector, and the first text feature vector is converted into a probability sequence to obtain the negative emotion probability corresponding to the comment text. If the negative emotion probability is less than or equal to the second preset threshold, the comment text is published; if the negative emotion probability is greater than the second preset threshold, the comment text is determined to be an insulting comment text. Therefore, the use of this application can effectively avoid the misjudgment of text content with positive emotions and improve the recognition accuracy of insulting content.

[0170] Based on the description of the above method embodiment, the embodiment of the present application further provides an abusive content identification device, which can be a computer program (including program code) running in a computer device, for example, the abusive content identification device is an application software; the device can be used to execute the corresponding steps of the method provided in the embodiment of the present application. Figure 5 , is a schematic diagram of the structure of an insult content identification device provided in an embodiment of the present application. Figure 5 As shown, the insult content identification device 5 may include: a judgment module 51 , a determination and extraction module 52 , an insult probability calculation module 53 , a negative emotion probability calculation module 54 and a determination module 55 .

[0171] A judging module 51, used for judging whether a plurality of comment words in the comment text contain any insulting keyword in the insulting text library;

[0172] A determination extraction module 52 is used to extract a semantic feature matrix of the comment text if the comment text contains any of the insulting keywords;

[0173] An insult probability calculation module 53, used to obtain the insult probability of the comment text according to the semantic feature matrix of the comment text;

[0174] A negative emotion probability calculation module 54 is used to obtain the negative emotion probability of the comment text according to the semantic feature matrix of the comment text and a first weight vector, wherein the first weight vector is trained based on the first sample comment set and the emotion classification label corresponding to each comment text in the first sample comment set, and the first weight vector includes the emotion classification weight of each comment word in the multiple comment words;

[0175] The determination module 55 is used to determine whether the comment text is an insulting comment text according to the insult probability and the negative emotion probability.

[0176] Optionally, the negative emotion probability calculation module 54 is specifically used to:

[0177] Obtaining a first text feature vector of the comment text according to the semantic feature matrix of the comment text and the first weight vector;

[0178] The first text feature vector is converted into a probability sequence to obtain the negative sentiment probability of the comment text.

[0179] Optionally, the determining and extracting module 52 includes:

[0180] A first initial matrix determining unit 521, configured to sort the initial word vectors of each comment word in the comment text according to the order in which each comment word appears in the comment text, so as to obtain an initial matrix of the comment text;

[0181] A text matrix calculation unit 522, configured to obtain a text matrix of the comment text according to the initial matrix of the comment text and an input weight matrix, wherein the input weight matrix includes a pre-trained input weight column vector of a preset length;

[0182] The feature extraction unit 523 is used to extract features from the text matrix to obtain a semantic feature matrix of the comment text.

[0183] Optionally, the determining and extracting module 52 further includes:

[0184] The second initial matrix determination unit 524 is used to sort the initial word vectors of each comment word in each comment text in the second sample comment set to obtain the initial matrix of each comment text, and then obtain the initial matrix of the second sample comment set, wherein the second sample comment set includes comment text a i , the comment text a i Include target review word b j , where i is a positive integer less than or equal to the total number of comment texts corresponding to the second sample comment set, and j is a positive integer less than or equal to the number of comment texts a i A positive integer corresponding to the total number of comment words;

[0185] The calculation unit 525 is used to obtain the comment text a according to the initial matrix, the initial input weight matrix and the initial output weight matrix of the second sample comment set. i Target review word b j The corresponding output vector is used to obtain the target comment word b according to the output vector j The predicted word vector of

[0186] The weight matrix adjustment unit 526 is used to adjust the weight matrix according to the target comment word b j The predicted word vector and the target review word b j The initial word vector of the input weight matrix and the output weight matrix are adjusted;

[0187] The input weight matrix determination unit 527 is used to adjust the target comment word b j The predicted word vector and the target review word b j When the initial word vector satisfies the first convergence condition, the adjusted initial input weight matrix is ​​determined as the input weight matrix.

[0188] Optionally, the insult probability calculation module 53 includes:

[0189] A second text feature calculation unit 531 is used to obtain a second text feature vector according to the semantic feature matrix of the comment text and the second weight vector, wherein the second weight vector includes the insult recognition weight of each comment word in the multiple comment words, and the weight of the insult comment words in the second weight vector is higher than the weight of the non-insult comment words;

[0190] The second probability conversion unit 532 is used to convert the second text feature vector into a probability sequence to obtain the insult probability of the comment text.

[0191] Optionally, the device further includes: a weight vector determination module 56.

[0192] The weight vector determination module 56 includes:

[0193] A semantic feature extraction unit 561 is used to extract a semantic feature matrix of each comment text in the third sample comment set to obtain a semantic feature matrix of the third sample comment set;

[0194] A text feature calculation unit 562, configured to obtain a text feature vector of the third sample comment set according to the semantic feature matrix of the third sample comment set and the first initial weight vector;

[0195] A predicted emotion label determination unit 563 is used to convert the text feature vector of the third sample comment set into a probability sequence to obtain a predicted emotion classification label of each comment text in the third sample comment set;

[0196] An adjusting unit 564, configured to adjust the first initial weight vector according to the predicted sentiment classification label and the sentiment classification label of each comment text in the third sample comment set;

[0197] The first weight vector determining unit 565 is configured to determine the adjusted first initial weight vector as the first weight vector when the predicted sentiment classification label and the sentiment classification label of the comment text in the third sample comment set meet the second convergence condition.

[0198] Optionally, the text feature vector of the third sample comment set includes the mth element X m , m is a positive integer less than or equal to the total number M of comment texts corresponding to the third sample comment set;

[0199] The predicted emotion label determination unit 563 includes:

[0200] The probability value calculation subunit 5631 is used to calculate the mth element X m The probability value Wherein, n is a positive integer greater than or equal to 1 and less than or equal to M;

[0201] The probability sequence determination subunit 5632 is used to obtain the probability value of each element in the text feature vector of the third sample comment set according to the probability value Sm, and obtain the probability sequence corresponding to the text feature vector of the third sample comment set according to the probability value of each element.

[0202] Optionally, the insulting comment text carries a user account corresponding to the insulting comment text;

[0203] The device also includes:

[0204] The generating and sending module 57 is used to generate a report message according to the insulting comment text and the user account, and send the report message to the network management platform.

[0205] It can be understood that the insult content identification device 5 is used to implement Figure 2 and Figure 4 Steps performed by the abusive content identification platform in the embodiment. Figure 5 For specific implementation methods and corresponding beneficial effects of the functional blocks included in the insult content identification device 5, please refer to the aforementioned Figure 2 and Figure 4 The specific introduction of the embodiments will not be repeated here.

[0206] Above Figure 5 The insult content identification device 5 in the embodiment shown can be Figure 6 The abusive content identification device can be implemented by the server 600 shown in FIG. Figure 6 , is a schematic diagram of the structure of a server provided in an embodiment of the present application. Figure 6 As shown, the above-mentioned insult content identification device 600 may include: one or more processors 601, memory 602 and transceiver 603. The above-mentioned processor 601, memory 602 and transceiver 603 are connected via a bus 604. Among them, the above-mentioned transceiver 603 is used to receive or send data, and the above-mentioned memory 602 is used to store a computer program, and the computer program includes program instructions; the processor 601 is used to execute the program instructions stored in the memory 602, and perform the following operations:

[0207] Determine whether multiple comment words in the comment text contain any insulting keyword in the insulting text library;

[0208] If the comment text contains any of the insulting keywords, extracting a semantic feature matrix of the comment text;

[0209] Calculating the insult probability of the target comment text according to the semantic feature matrix of the comment text;

[0210] Obtaining the negative sentiment probability of the target comment text according to the semantic feature matrix of the comment text and the first weight vector, wherein the first weight vector is trained based on the second sample comment set and the sentiment classification label corresponding to each comment text in the second sample comment set, and the first weight vector includes the sentiment classification weight of each comment word in the multiple comment words;

[0211] According to the insult probability and the negative emotion probability, it is determined whether the comment text is an insult comment text.

[0212] Optionally, the processor 601 obtains the negative sentiment probability of the comment text according to the semantic feature matrix of the comment text and the first weight vector, and specifically performs the following operations:

[0213] Obtaining a first text feature vector of the comment text according to the semantic feature matrix of the comment text and the first weight vector;

[0214] The first text feature vector is converted into a probability sequence to obtain the negative sentiment probability of the comment text.

[0215] Optionally, the processor 601 extracts the semantic features of the comment text by performing the following operations:

[0216] According to the order of appearance of each comment word in the comment text, the initial word vector of each comment word is sorted to obtain the initial matrix of the comment text;

[0217] Obtaining a text matrix of the comment text according to the initial matrix of the comment text and the input weight matrix, wherein the input weight matrix includes pre-trained input weight column vectors of a preset length;

[0218] Feature extraction is performed on the text matrix to obtain a semantic feature matrix of the comment text.

[0219] Optionally, before obtaining the text matrix of the comment text according to the initial matrix of the comment text and the input weight matrix, the processor 601 specifically performs the following operations:

[0220] The initial word vectors of each comment word in each comment text in the second sample comment set are sorted to obtain the initial matrix of each comment text, and then the initial matrix of the second sample comment set is obtained, wherein the second sample comment set includes comment text a i , the comment text ai Include target review word b j , where i is a positive integer less than or equal to the total number of comment texts corresponding to the second sample comment set, and j is a positive integer less than or equal to the number of comment texts a i A positive integer corresponding to the total number of comment words;

[0221] The comment text a is obtained according to the initial matrix, the initial input weight matrix and the initial output weight matrix of the second sample comment set. i Target review word b j The corresponding output vector is used to obtain the target comment word b according to the output vector j The predicted word vector of

[0222] According to the target review word b j The predicted word vector and the target review word b j The initial word vector of , adjusting the initial input weight matrix and the output weight matrix;

[0223] When the adjusted target review word b j The predicted word vector and the target review word b j When the initial word vector satisfies the first convergence condition, the adjusted initial input weight matrix is ​​determined as the input weight matrix.

[0224] Optionally, the processor 601 calculates the insult probability of the comment text according to the semantic feature matrix of the comment text, and specifically performs the following operations:

[0225] Obtaining a second text feature vector according to the semantic feature matrix of the comment text and the second weight vector, wherein the second weight vector includes the insult recognition weight of each comment word in the plurality of comment words, and the weight of the insult comment words in the second weight vector is higher than the weight of the non-insult comment words;

[0226] The second text feature vector is converted into a probability sequence to obtain the insult probability of the comment text.

[0227] Optionally, the processor 601 further performs the following operations:

[0228] Extracting a semantic feature matrix of each comment text in the third sample comment set to obtain a semantic feature matrix of the third sample comment set;

[0229] Obtaining a text feature vector of the third sample comment set according to the semantic feature matrix of the third sample comment set and the first initial weight vector;

[0230] Converting the text feature vector of the third sample comment set into a probability sequence to obtain a predicted sentiment classification label of each comment text in the third sample comment set;

[0231] The first initial weight vector is adjusted according to the predicted sentiment classification label and sentiment classification label of each comment text in the third sample comment set; wherein the text feature vector of the third sample comment set includes the mth element X m , m is a positive integer less than or equal to the total number M of comment texts corresponding to the third sample comment set;

[0232] When the predicted sentiment classification labels and sentiment classification labels of the comment texts in the third sample comment set meet the second convergence condition, the adjusted first initial weight vector is determined as the first weight vector.

[0233] Optionally, the processor 601 converts the text feature vector of the third sample comment set into a probability sequence by performing the following operations:

[0234] Calculate the mth element X m The probability value Wherein, n is a positive integer greater than or equal to 1 and less than or equal to M;

[0235] The probability value of each element in the text feature vector of the third sample comment set is obtained according to the probability value Sm, and the probability sequence corresponding to the text feature vector of the third sample comment set is obtained according to the probability value of each element.

[0236] Optionally, the insulting comment text carries a user account corresponding to the insulting comment text;

[0237] The processor 601 further performs the following operations:

[0238] A report message is generated according to the insulting comment text and the user account, and the report message is sent to a network management platform.

[0239] In addition, it should be pointed out that: the embodiment of the present application further provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the aforementioned abusive content identification device 5, and the computer program includes program instructions. When the processor executes the program instructions, the aforementioned abusive content identification device 5 can be executed. Figure 2 or Figure 4The description of the above-mentioned method for identifying abusive content in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, program instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one location, or on multiple computing devices distributed at multiple locations and interconnected by a communication network. Multiple computing devices distributed at multiple locations and interconnected by a communication network can constitute a blockchain system.

[0240] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the above-mentioned program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the above-mentioned storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0241] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the functions specified in the process. Figure 1A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0242] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A method for identifying insulting content, characterized in that: include: Determine whether multiple comment words in the comment text contain any insulting keyword in the insulting text library; If the comment text contains any of the insulting keywords, extracting a semantic feature matrix of the comment text; Obtaining a second text feature vector according to the semantic feature matrix of the comment text and the second weight vector, wherein the second weight vector includes an insult recognition weight of each of the plurality of comment words, and the weight of the insult comment words in the second weight vector is higher than the weight of the non-insult comment words; Converting the second text feature vector into a probability sequence to obtain a probability of insult in the comment text; Obtaining the negative sentiment probability of the comment text according to the semantic feature matrix of the comment text and the first weight vector, wherein the first weight vector is obtained by training based on the first sample comment set and the sentiment classification label corresponding to each comment text in the first sample comment set, and the first weight vector includes the sentiment classification weight of each comment word in the multiple comment words; According to the insult probability and the negative emotion probability, it is determined whether the comment text is an insult comment text.

2. The method according to claim 1, characterized in that The step of obtaining the negative sentiment probability of the comment text according to the semantic feature matrix of the comment text and the first weight vector includes: Obtaining a first text feature vector of the comment text according to the semantic feature matrix of the comment text and the first weight vector; The first text feature vector is converted into a probability sequence to obtain the negative sentiment probability of the comment text.

3. The method according to claim 1, characterized in that The step of extracting the semantic feature matrix of the comment text includes: According to the order of appearance of each comment word in the comment text, the initial word vector of each comment word is sorted to obtain the initial matrix of the comment text; Obtaining a text matrix of the comment text according to the initial matrix of the comment text and the input weight matrix, wherein the input weight matrix includes pre-trained input weight column vectors of a preset length; Feature extraction is performed on the text matrix to obtain a semantic feature matrix of the comment text.

4. The method according to claim 3, before obtaining the text matrix of the text according to the initial matrix of the comment text and the input weight matrix, comprises: The initial word vectors of each comment word in each comment text in the second sample comment set are sorted to obtain the initial matrix of each comment text, and then the initial matrix of the second sample comment set is obtained, wherein the second sample comment set includes comment text a i , the comment text a i Include target review word b j , where i is a positive integer less than or equal to the total number of comment texts corresponding to the second sample comment set, and j is a positive integer less than or equal to the number of comment texts a i A positive integer corresponding to the total number of comment words; The comment text a is obtained according to the initial matrix, the initial input weight matrix and the initial output weight matrix of the second sample comment set. i Target review word b j The corresponding output vector is used to obtain the target comment word b according to the output vector j The predicted word vector of According to the target review word b j The predicted word vector and the target review word b j The initial word vector of the input weight matrix and the output weight matrix are adjusted; When the adjusted target review word b j The predicted word vector and the target review word b j When the initial word vector satisfies the first convergence condition, the adjusted initial input weight matrix is ​​determined as the input weight matrix.

5. The method according to claim 1, characterized in that: Before obtaining the first text feature vector of the comment text according to the semantic feature matrix of the comment text and the first weight vector, the method further includes: Extracting a semantic feature matrix of each comment text in the third sample comment set to obtain a semantic feature matrix of the third sample comment set; Obtaining a text feature vector of the third sample comment set according to the semantic feature matrix of the third sample comment set and the first initial weight vector; Converting the text feature vector of the third sample comment set into a probability sequence to obtain a predicted sentiment classification label of each comment text in the third sample comment set; Adjusting the first initial weight vector according to the predicted sentiment classification label and the sentiment classification label of each comment text in the third sample comment set; When the predicted sentiment classification labels and sentiment classification labels of the comment texts in the third sample comment set meet the second convergence condition, the adjusted first initial weight vector is determined as the first weight vector.

6. The method according to claim 5, characterized in that The text feature vector of the third sample comment set includes the mth element X m , m is a positive integer less than or equal to the total number M of comment texts corresponding to the third sample comment set; The converting the text feature vector of the third sample comment set into a probability sequence comprises: Calculate the mth element X m The probability value Sm= , wherein n is an integer greater than or equal to 1 and less than or equal to M; The probability value of each element in the text feature vector of the third sample comment set is obtained according to the probability value Sm, and the probability sequence corresponding to the text feature vector of the third sample comment set is obtained according to the probability value of each element.

7. The method according to claim 1, characterized in that The insulting comment text carries the user account corresponding to the insulting comment text; After determining that the comment text is an insulting comment text, the method further includes: A report message is generated according to the insulting comment text and the user account, and the report message is sent to a network management platform.

8. A device for identifying insulting content, characterized in that: include: A judgment module, used for judging whether a plurality of comment words in the comment text contain any insulting keyword in the insulting text library; determining an extraction module, for extracting a semantic feature matrix of the comment text if the comment text contains any of the insulting keywords; an insult probability calculation module, configured to obtain a second text feature vector according to the semantic feature matrix of the comment text and the second weight vector, wherein the second weight vector includes an insult recognition weight of each comment word in the plurality of comment words, and the weight of the insult comment words in the second weight vector is higher than the weight of the non-insult comment words; convert the second text feature vector into a probability sequence to obtain an insult probability of the comment text; A negative emotion probability calculation module, used to obtain the negative emotion probability of the comment text according to the semantic feature matrix of the comment text and a first weight vector, wherein the first weight vector is trained based on the first sample comment set and the emotion classification label corresponding to each comment text in the first sample comment set, and the first weight vector includes the emotion classification weight of each comment word in the multiple comment words; The determination module is used to determine whether the comment text is an insulting comment text according to the insult probability and the negative emotion probability.

9. A server, characterized in that: It comprises a processor, a memory and a transceiver, wherein the processor, the memory and the transceiver are connected to each other, wherein the transceiver is used to receive or send data, the memory is used to store program code, and the processor is used to call the program code to execute the method for identifying abusive content according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Entity-level affective classification method based on convolution attention mechanism network

    CN109213868A

  • Comment analysis method based on word vectors and syntactic features and visual interactive interface

    CN110175325A

  • Comment text processing method and device, storage medium and computer device

    CN110866398A

  • Emotion analysis method and device, electronic equipment and storage medium

    CN111291187A

  • Semisupervised autoencoder for sentiment analysis

    US20180165554A1