Text sentiment classification method and device

Abstract

The present invention discloses a text sentiment classification method and device, and relates to the field of computer technology. A specific implementation of the method includes: obtaining a text to be classified; determining the word features, phrase features and sentence features of the text to be classified, fusing the word features, phrase features and sentence features to obtain the target features of the text to be classified; determining the sentiment of the text to be classified according to the target features of the text to be classified and a pre-trained sentiment classification model. This implementation saves a lot of work on manually annotating data and reduces labor costs; based on the word features, phrase features and sentence features obtained by the analysis model of three granularities, a multi-level sentiment feature about the text is obtained by fusing the three features, and then the sentiment of the text is determined through a multi-level sentiment classification model and the multi-level sentiment feature, thereby improving the accuracy of text classification.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a text sentiment classification method and device. Background Art

[0002] At present, the commonly used text sentiment classification method is to analyze sentiment by counting sentiment words. This type of method uses sentiment dictionaries as the main basis for judging sentiment tendencies, uses dictionaries for sentiment analysis, and counts and weights the sentiment-related words that have been evaluated and marked. However, sentiment dictionaries are manually constructed. Therefore, the accuracy of this type of method is limited by the quality of the sentiment dictionary on the one hand, and is easily affected by local content on the other hand. Existing analysis methods either divide text sentiment into two polarities, positive and negative, for analysis, or finely divide the sentiment of the text to construct a multivariate sentiment analysis method. However, classifying text sentiment into two categories, positive and negative, can only obtain the positive and negative sentiment tendencies in the text, and cannot analyze the sentiment of text without strong sentiment tendencies. The classification results of this type of method that finely divides text sentiment are prone to confusion, and different sentiment types may be similar texts in text representation. For example, questioning and suspicion are very close in text features, but these two emotions contain different sentiment tendencies. Questioning tends to be negative, while suspicion tends to be neutral, and the emotions conveyed are very different. If sentiment analysis is performed directly on short texts, it is easy to confuse these two types of emotions, resulting in inaccurate text sentiment classification. The combination of different words in Chinese text will convey different subjective emotions. Therefore, if sentiment classification is only performed on the word granularity in the text, and sentiment classification based on higher granularity in the word granularity in the text is not performed, the accuracy of the obtained text sentiment classification result will be low. Summary of the invention

[0003] In view of this, the embodiments of the present invention provide a text sentiment classification method and device, which saves a lot of work on manual data annotation and reduces labor costs; based on the word features, phrase features and sentence features obtained by the analysis model of three granularities, a multi-level sentiment feature about the text is obtained by fusing the three features, and then the sentiment of the text is determined through a multi-level sentiment classification model and the multi-level sentiment feature, thereby improving the accuracy of text classification.

[0004] To achieve the above object, according to one aspect of an embodiment of the present invention, a text sentiment classification method is provided, comprising:

[0005] Get the text to be classified;

[0006] Determine the word features, phrase features and sentence features of the text to be classified, and fuse the word features, phrase features and sentence features to obtain the target features of the text to be classified;

[0007] The sentiment of the text to be classified is determined according to the target features of the text to be classified and a pre-trained sentiment classification model.

[0008] Optionally, determining the word features, phrase features, and sentence features of the text to be classified includes:

[0009] Determining the word features of the text to be classified through a pre-trained word analysis model;

[0010] Determining the phrase features of the text to be classified by using a pre-trained phrase analysis model;

[0011] The sentence features of the text to be classified are determined by using a pre-trained sentence analysis model.

[0012] Optionally, determining the word features, phrase features, and sentence features of the text to be classified includes:

[0013] Performing word segmentation processing on the text to be classified to obtain a plurality of words to be analyzed;

[0014] Determining word vectors of the multiple words to be analyzed;

[0015] Determining the word features of the text to be classified according to the word vector and the pre-trained word analysis model;

[0016] Determining phrase features of the text to be classified according to the word vector and the pre-trained phrase analysis model;

[0017] The sentence features of the text to be classified are determined based on the word vector and the pre-trained sentence analysis model.

[0018] Optionally, the pre-trained word analysis model includes a first convolutional layer and a first attention module.

[0019] Optionally, determining the phrase features of the text to be classified according to the word vector and the pre-trained phrase analysis model includes:

[0020] The word vectors of any two consecutive words to be analyzed are input into the pre-trained phrase analysis model to determine the phrase features of the text to be classified.

[0021] Optionally, the pre-trained phrase analysis model includes a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a second attention module, a third attention module and a global maximum pooling layer.

[0022] Optionally, determining the phrase features of the text to be classified according to the word vector and the pre-trained phrase analysis model includes:

[0023] Input the word vectors of any two consecutive words to be analyzed into the fourth convolutional layer to obtain a first convolutional feature, and input the first convolutional feature into the second attention module to obtain a first phrase feature;

[0024] Inputting the first convolution feature into the fifth convolution layer to obtain a second convolution feature, and inputting the second convolution feature into the third attention module to obtain a second phrase feature;

[0025] Inputting the second convolution feature into the sixth convolution layer to obtain a third convolution feature, and inputting the third convolution layer feature into the global maximum pooling layer to obtain a third phrase feature;

[0026] The first phrase feature, the second phrase feature and the third phrase feature are integrated to obtain the phrase feature of the text to be classified.

[0027] Optionally, the pre-trained sentence analysis model includes a seventh convolutional layer and an LSTM layer.

[0028] Optionally, before performing word segmentation processing on the text to be classified, the method further includes: performing denoising processing and stop word removal processing on the text to be classified according to preset rules.

[0029] Optionally, the pre-trained sentiment classification model is trained according to the following process:

[0030] Obtain training sample data;

[0031] Determine the parent emotion category and the first classification loss of the training sample data by using the pre-trained first classification model;

[0032] Determine the subclass sentiment category and the second classification loss of the training sample data by using the pre-trained second classification model;

[0033] Constructing a target loss function according to the first classification loss and the second classification loss;

[0034] The pre-trained sentiment classification model is trained according to the target loss function.

[0035] Optionally, the parent emotion category includes positive, negative and neutral; the child emotion category includes at least one or more of the following: like, happiness, surprise, anger, fear, disgust, sadness, and neutral.

[0036] Optionally, constructing a target loss function according to the first classification loss and the second classification loss includes:

[0037] Determining a first product of the first classification loss and a preset weight coefficient;

[0038] Determine a difference between 1 and the weight coefficient, and determine a second product of the second classification loss and the difference;

[0039] Based on the first product and the second product, a target loss function is constructed.

[0040] To achieve the above object, according to another aspect of an embodiment of the present invention, a text sentiment classification device is provided, comprising:

[0041] A text acquisition module is used to acquire the text to be classified;

[0042] A feature determination module, used to determine the word features, phrase features and sentence features of the text to be classified, and to fuse the word features, phrase features and sentence features to obtain the target features of the text to be classified;

[0043] The emotion determination module is used to determine the emotion of the text to be classified according to the target features of the text to be classified and the pre-trained emotion classification model.

[0044] Optionally, the feature determination module is further used to:

[0045] Determining the word features of the text to be classified through a pre-trained word analysis model;

[0046] Determining the phrase features of the text to be classified by using a pre-trained phrase analysis model;

[0047] The sentence features of the text to be classified are determined by using a pre-trained sentence analysis model.

[0048] Optionally, the feature determination module is further used to:

[0049] Performing word segmentation processing on the text to be classified to obtain a plurality of words to be analyzed;

[0050] Determining word vectors of the multiple words to be analyzed;

[0051] Determining the word features of the text to be classified according to the word vector and the pre-trained word analysis model;

[0052] Determining phrase features of the text to be classified according to the word vector and the pre-trained phrase analysis model;

[0053] The sentence features of the text to be classified are determined based on the word vector and the pre-trained sentence analysis model.

[0054] Optionally, the pre-trained word analysis model includes a first convolutional layer and a first attention module.

[0055] Optionally, the feature determination module is further used to: input word vectors of any two consecutive words to be analyzed into the pre-trained phrase analysis model to determine the phrase features of the text to be classified.

[0056] Optionally, the pre-trained phrase analysis model includes a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a second attention module, a third attention module and a global maximum pooling layer.

[0057] Optionally, the feature determination module is further used to:

[0058] Input the word vectors of any two consecutive words to be analyzed into the fourth convolutional layer to obtain a first convolutional feature, and input the first convolutional feature into the second attention module to obtain a first phrase feature;

[0059] Inputting the first convolution feature into the fifth convolution layer to obtain a second convolution feature, and inputting the second convolution feature into the third attention module to obtain a second phrase feature;

[0060] Inputting the second convolution feature into the sixth convolution layer to obtain a third convolution feature, and inputting the third convolution layer feature into the global maximum pooling layer to obtain a third phrase feature;

[0061] The first phrase feature, the second phrase feature and the third phrase feature are integrated to obtain the phrase feature of the text to be classified.

[0062] Optionally, the pre-trained sentence analysis model includes a seventh convolutional layer and an LSTM layer.

[0063] Optionally, the feature determination module is further used to: before performing word segmentation processing on the text to be classified, perform denoising processing and stop word removal processing on the text to be classified according to preset rules.

[0064] Optionally, the device further comprises a model training module, which is used to:

[0065] Obtain training sample data;

[0066] Determine the parent emotion category and the first classification loss of the training sample data by using the pre-trained first classification model;

[0067] Determine the subclass sentiment category and the second classification loss of the training sample data by using the pre-trained second classification model;

[0068] Constructing a target loss function according to the first classification loss and the second classification loss;

[0069] The pre-trained sentiment classification model is trained according to the target loss function.

[0070] Optionally, the parent emotion category includes positive, negative and neutral; the child emotion category includes at least one or more of the following: like, happiness, surprise, anger, fear, disgust, sadness, and neutral.

[0071] Optionally, the model training module is also used to: determine a first product of the first classification loss and a preset weight coefficient; determine a difference between 1 and the weight coefficient, and determine a second product of the second classification loss and the difference; and construct a target loss function based on the first product and the second product.

[0072] To achieve the above-mentioned purpose, according to another aspect of an embodiment of the present invention, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the text sentiment classification method of an embodiment of the present invention.

[0073] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the text sentiment classification method of the embodiment of the present invention is implemented.

[0074] An embodiment of the above invention has the following advantages or beneficial effects: because the target features of the text to be classified are obtained by determining the word features, phrase features and sentence features of the text to be classified and fusing the word features, phrase features and sentence features; the technical means of determining the emotion of the text to be classified is used according to the target features of the text to be classified and the pre-trained sentiment classification model, thereby saving a lot of work of manually annotating data and reducing labor costs; based on the word features, phrase features and sentence features obtained by the analysis model of three granularities, multi-level sentiment features about the text are obtained by fusing the three features, and then the sentiment of the text is determined through the multi-level sentiment classification model and the multi-level sentiment features, thereby improving the accuracy of text classification.

[0075] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0077] Figure 1 Schematic diagram of the main process of the text sentiment classification method according to an embodiment of the present invention;

[0078] Figure 2 Schematic diagram of a word analysis model of a text sentiment classification method according to an embodiment of the present invention;

[0079] Figure 3 Schematic diagram of a phrase analysis model of a text sentiment classification method according to an embodiment of the present invention;

[0080] Figure 4 Schematic diagram of a sentence analysis model of a text sentiment classification method according to an embodiment of the present invention;

[0081] Figure 5 Schematic diagram of a sub-process of a text sentiment classification method according to an embodiment of the present invention;

[0082] Figure 6 It is a schematic diagram of determining multi-level sentiment classification of training samples in the text sentiment classification method of an embodiment of the present invention;

[0083] Figure 7 Schematic diagram of main modules of the text sentiment classification device according to an embodiment of the present invention;

[0084] Figure 8 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0085] Fig. 9 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0086] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0087] Figure 1 is a flow chart of the main steps of the text sentiment classification method according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0088] Step S101: obtaining text to be classified;

[0089] Step S102: determining the word features, phrase features and sentence features of the text to be classified, and integrating the word features, phrase features and sentence features to obtain the target features of the text to be classified;

[0090] Step S103: Determine the sentiment of the text to be classified according to the target features of the text to be classified and the pre-trained sentiment classification model.

[0091] For step S102, words are usually regarded as the smallest component unit in a text. The information of words in a text plays a very important role in text sentiment classification. If words with prominent emotional colors can be mined from a text, it will be of great help to text sentiment classification. Therefore, this embodiment needs to extract word features of the text to be classified to analyze the sentiment of the text to be classified.

[0092] However, the words in the text often do not exist independently. In many cases, it is necessary to connect the words before and after to more accurately express the emotions that the author wants to express. If only the words in the sentence are analyzed, it may not be possible to obtain the emotions conveyed by the phrases in the sentence. For example, when the sentence contains phrases such as "not angry" and "it's hard to be happy", if only the emotions conveyed by each word are analyzed, then "not angry" will be mainly analyzed according to the emotion type contained in "angry", and "it's hard to be happy" will most likely be considered to belong to the emotion type contained in "happy". In order to avoid the above errors, when analyzing the emotions of the text to be classified, this embodiment needs to consider the phrase information of two consecutive words in the text to be classified, that is, it is necessary to extract the phrase features of the text to be classified. By analyzing two consecutive words, this embodiment can obtain more comprehensive emotional information that is closer to the actual expression of the text.

[0093] A text containing multiple consecutive words has certain correlations between the content information before and after it. It is difficult to describe the entire text by only mining information on part of the text. In order to describe the complete text, this embodiment also needs to extract sentence features of the text to be classified.

[0094] In an optional embodiment, this step can determine the word features of the text to be classified through a pre-trained word analysis model; determine the phrase features of the text to be classified through a pre-trained phrase analysis model; and determine the sentence features of the text to be classified through a pre-trained sentence analysis model. Among them, the word analysis model and the phrase analysis model adopt a convolutional neural network model, and the sentence analysis model adopts a convolutional neural network model and a long short-term memory neural network model. In this embodiment, the word analysis model, the phrase analysis model and the sentence analysis model are constructed using a convolutional neural network model, which solves the problem of the need to manually annotate a large training data set in the prior art, because the convolutional neural network model does not rely too much on prior data and has no rigid requirements for manually annotated data. Therefore, the text sentiment classification method of the embodiment of the present invention saves a lot of work on manually annotating data.

[0095] In an optional embodiment, after obtaining the text to be classified and before determining the word features, phrase features and sentence features of the text to be classified, the text to be classified needs to be preprocessed. Specifically, the preprocessing of the text to be classified includes denoising, word segmentation and stop word removal.

[0096] Among them, the denoising of the text to be analyzed mainly refers to filtering out useless information and meaningless characters from the original text to be classified, thereby extracting meaningful information from the text to be classified. As a specific example, historical corpus can be manually analyzed, and then rules or mechanisms can be established. Finally, useless information and meaningless characters in the text to be classified can be filtered out through the established rules or mechanisms, thereby extracting valuable data content.

[0097] The word segmentation process mainly divides a sentence into a number of unrelated words. As a specific example, the present embodiment can use the jieba library in Python to perform word segmentation on the text to be classified.

[0098] After word segmentation, there are stop words and other meaningless words in the multiple words to be analyzed, so it is necessary to remove the stop words in the multiple words to be analyzed.

[0099] After preprocessing the text to be classified, the words to be analyzed are vectorized, that is, the word vectors of the words to be analyzed are determined. The machine cannot directly recognize the content in the text to be classified, so it is necessary to select the features in the text to be classified, and then convert them into a form that the machine can understand, that is, use vectors to represent text feature items, which is also called word vectorization. After the text is processed by denoising, word segmentation, and stop words, it is necessary to convert the text from an unstructured form to a structured vector form for representation, so that the text can be further analyzed and processed in a form that the computer can understand. As a specific example, this embodiment uses the Word2vec toolkit to vectorize the preprocessed text to be classified. Among them, Word2vec is a related model for generating word vectors. This embodiment uses Word2vec to convert each word to be analyzed in the text to be classified into an m-dimensional word vector form. For a text A to be classified containing T words, the text to be classified can be represented as a T×m matrix after conversion into a word vector. The text A to be classified is represented as:

[0100] A=[a1,a2,…a t …a T ]

[0101] Among them, a t It is the word vector corresponding to the tth word to be analyzed, with a dimension of m.

[0102] After determining the word vector of the word to be analyzed, the word features of the text to be classified can be determined based on the word vector and the pre-trained word analysis model; the phrase features of the text to be classified can be determined based on the word vector and the pre-trained phrase analysis model; and the sentence features of the text to be classified can be determined based on the word vector and the pre-trained sentence analysis model.

[0103] Among them, Figure 2 As shown, the pre-trained word analysis model includes a first convolutional layer and a first attention module.

[0104] In order to obtain deeper feature information from the word vector, this embodiment constructs a convolution layer, and obtains relatively abstract information in the word vector through the calculation of the convolution layer. The expression of the convolution layer is shown in the following formula (1):

[0105] X c =relu(w c0 *A+b c0 ) (1)

[0106] Among them, w c0 is the weight matrix of the convolutional layer, b c0 is the bias matrix of the convolutional layer. Through this formula, we get the convolution feature representation X of each word in the sentence c In this embodiment, the weight matrix and the bias matrix may be initialized using the MSRA method, and the weight matrix and the bias matrix may be continuously updated and iteratively optimized through model training.

[0107] In the classification method based on neural network, a fully connected layer is usually constructed as a classifier at the last layer of the network, and the probability of each category is output through the softmax function. However, since the weight dimension of the fully connected layer is fixed, the feature dimension of the fully connected layer input also needs to be fixed. In order to obtain the feature representation of the entire sentence with a fixed dimension, the existing methods usually use global average pooling to pool the feature sequence of a sentence into a vector with a fixed dimension, as shown in the following formula (2).

[0108]

[0109] in, is the feature vector corresponding to the tth word to be analyzed, Y pooling Refers to a fixed-dimensional vector of sentence features after global average pooling. However, most words in a sentence are useless for sentiment analysis, and some may even cause noise interference. However, using the global average pooling method to add all the features of a sentence together will not highlight the value of words that can represent the sentiment tendency of the entire sentence, and may also introduce noise interference.

[0110] In order to highlight the value of words with emotional information in sentences, this embodiment constructs an attention module in the word analysis model. The principle of the attention mechanism is to distinguish the contributions of different features and highlight more important features by calculating the attention weights of features at different positions. In this embodiment, the attention weight is calculated using two convolution operations and one softmax operation, as shown in the following formula (3):

[0111] M c =softmax(w c2 (w c1 *X c +b c1 )) (3)

[0112] Among them, w c1 and b c1 are the weight matrix and bias matrix of the first convolutional layer in the attention module, w c2 is the weight matrix of the second convolutional layer, soft max() represents the softmax activation function, M c is the attention weight.

[0113] By calculating the attention weights of the words to be analyzed, on the one hand, the value of words that contribute greatly to the text sentiment classification task can be highlighted, and on the other hand, the value of words that contribute little to the text sentiment classification task can be suppressed.

[0114] After obtaining the attention weight of the word to be analyzed, the word feature of the word to be analyzed is obtained according to the following formula (4):

[0115]

[0116] Y c is the phrase feature of the word to be analyzed, is the attention weight of the tth word to be analyzed, is the convolution feature of the word to be analyzed.

[0117] like Figure 3 As shown, the pre-trained phrase analysis model includes a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a second attention module, a third attention module and a global maximum pooling layer. The input of the pre-trained phrase analysis model is the word vectors of two consecutive words to be analyzed.

[0118] In order to obtain deep phrase information from the text to be classified A, this embodiment constructs a convolution layer, namely the fourth convolution layer. When performing convolution calculation, the fourth convolution layer takes the word vectors of two consecutive words to be analyzed as input each time, thereby obtaining the phrase convolution feature X of the two consecutive words to be analyzed. d, as shown in formula (5).

[0119] X d =relu(w d0 *A+b d0 ) (5)

[0120] Phrases at different positions in a sentence contain different information, and important sentiment information often only exists in some positions in a sentence. In order to highlight phrases with greater value contribution in a sentence, this embodiment constructs an attention module, which includes two convolution operations and one softmax operation, as shown in formula (6).

[0121] M d =softmax(w d2 (w d1 *X d +b d1 )) (6)

[0122] Among them, w d1 and b d1 are the weight matrix and bias matrix of the first convolutional layer in the attention module, w d2 is the weight matrix of the second convolutional layer, w d2 ×b d1 is the bias matrix of the second convolutional layer, M d is the attention weight. The attention weight reflects the importance of phrase features in the sentiment classification task. Phrase features with high attention weights play a greater role in sentiment classification. Further, the phrase convolution features are fused using the attention weights to obtain phrase features, as shown in formula (7).

[0123]

[0124] By introducing the attention mechanism, the attention features of the phrase can better reflect the emotional information in the sentence.

[0125] However, the above process only considers phrases consisting of two consecutive words. In practice, due to the wide variety of text grammars, a complete phrase may be represented by multiple words; on the other hand, due to the limitations of word segmentation technology, a complete phrase may be divided into multiple words. Therefore, only considering the case where two consecutive words form a phrase may miss key information.

[0126] In order to capture features at different levels, this embodiment builds a phrase analysis model with a multi-layer pyramid structure on the basis of the above. Figure 3As shown. In the phrase analysis model, the word vectors of any two consecutive words to be analyzed are input into the fourth convolution layer to obtain a first convolution feature, and the first convolution feature is input into the second attention module to obtain a first phrase feature; the first convolution feature is input into the fifth convolution layer to obtain a second convolution feature, and the second convolution feature is input into the third attention module to obtain a second phrase feature; the second convolution feature is input into the sixth convolution layer to obtain a third convolution feature, and the third convolution layer feature is input into the global maximum pooling layer to obtain a third phrase feature; the first phrase feature, the second phrase feature and the third phrase feature are fused to obtain the phrase feature of the text to be classified.

[0127] like Figure 4 As shown, the pre-trained sentence analysis model includes the seventh convolutional layer and the LSTM layer. The input of the sentence analysis model is the word vector of all the words to be analyzed in the text to be classified.

[0128] The sentence analysis model first passes the input word vector matrix through a seventh convolutional layer that captures phrase information. The purpose of passing through the seventh convolutional layer is to obtain the abstract features of the word vector. After the seventh convolutional layer, an LSTM layer (LSTM stacked neural network) is constructed, and the input of the LSTM layer is the convolutional features output by the seventh convolutional layer. The calculation process of each "gate" in the LSTM layer is shown in the following formula:

[0129]

[0130]

[0131]

[0132] s t =f t *s t-1 +i t *g t

[0133] o t =σ(W xo x t +W ho h t-1 +b o )

[0134] h t =o t tanh(s t )

[0135] Among them, i t Is the state of the input gate, which determines the input of the network at the current moment How much is saved to the cell state s t,s t is the unit state at the current moment, which consists of two parts: one is to discard some long-term memory information through the forget gate; the other is to add the newly added information at the current moment through the input gate. t is the output gate, which determines the empty state unit s t How many outputs are there to the current output value h of the LSTM t ,h t is the output of the LSTM unit at the current moment, The input of the j-th layer word at the current moment, σ represents the sigmoid activation function, h t-1 is the output of the LSTM unit at the previous moment, W xi , W hi and b i The network parameters of the LSTM layer input gate; f t is the state of the forget gate, which determines the unit state s t-1 The degree of discard, W xf , W hf and b f is the network parameter of the forget gate of the LSTM layer; g t is the candidate information, which is used to calculate the candidate unit state of the current input, W xg , W hg and b g is the network parameter of the LSTM layer memory unit, W xo , W ho and b o is the network parameter of the LSTM layer output gate, tanh is the hyperbolic tangent activation function, and * indicates bitwise multiplication of the elements in the vector. The forget gate is used to calculate the information to be forgotten, the input gate is used to calculate the information saved in the state unit, and the output gate calculates the information to be output through the sigmoid function, and then multiplies it by the value of the current unit state through the tanh function to obtain the output. In order to describe the information of the entire sentence, this embodiment uses the state feature h of the last LSTM output as the feature Y of the entire sentence j =h.

[0136] After obtaining the word features, phrase features and sentence features of the text to be classified, the word features, phrase features and sentence features are fused in series to obtain the target features of the text to be classified.

[0137] Chinese text has no fixed format when conveying emotions. Therefore, different text granularities will convey different emotions. Therefore, representing Chinese text features based on different text granularities will affect the sentiment classification results of the text. The embodiment of the present invention divides Chinese sentences into three granularities: words, phrases, and sentences. For different text granularities, the embodiment of the present invention constructs word analysis models respectively. The phrase analysis model and the sentence analysis model are used to capture the sentiment information at the word, phrase, and sentence levels respectively. By fusing the sentiment features of three different levels in a concatenated feature manner, a multi-level sentiment feature representation of the text is obtained, which is more accurate for the representation of text features.

[0138] For step S103, after obtaining the multi-level sentiment features of the text to be classified, the sentiment of the text to be classified is determined using the multi-level sentiment classification model, ie, the pre-trained sentiment classification model.

[0139] Specifically, Figure 5 As shown, the pre-trained sentiment classification model is trained according to the following process:

[0140] Step S501: Obtain training sample data;

[0141] Step S502: using the pre-trained first classification model, determining the parent emotion category and the first classification loss of the training sample data;

[0142] Step S503: using the pre-trained second classification model, determining the subclass emotion category and the second classification loss of the training sample data;

[0143] Step S504: constructing a target loss function according to the first classification loss and the second classification loss;

[0144] Step S505: According to the target loss function, the pre-trained sentiment classification model is trained.

[0145] For step S502, Figure 6 As shown, the parent emotion category of the training sample is first analyzed by the first classification model (large category emotion classification model), and then the sub-category emotion category of the training sample data is determined by the pre-trained second classification model (small category emotion classification model), so as to perform deep emotion analysis on the training sample. In this embodiment, in order to describe the emotion of the text in more detail, the text emotion is divided into three parent emotion categories: positive, negative and neutral, and eight sub-emotion categories: like, happiness, surprise, anger, fear, disgust, sadness, and none (i.e., like, happiness, surprise, anger, fear, disgust, sadness, and neutral).

[0146] In the classification process, the first classification model is first used to calculate the major category sentiment, as shown in the following formula (8).

[0147]

[0148] Where p(c=dalei|T n ) represents the probability that the nth text belongs to the daleith category of emotion, which is the output value of the first classification model. c represents the probability that the text belongs to the cth category, Y represents the target feature, b represents the bias term, and w c Y+b is a linear text classification function, exp(w c Y+b) indicates the index of the text belonging to the cth category, ∑ c exp(w c Y+b) represents the index sum of the text belonging to the cth category. The index is taken for normalized mapping. Then, the first classification loss is calculated using the parent sentiment category result, as shown in the following formula (9).

[0149] L da =-y n log(p(c=dalei|T n )) (9)

[0150] Among them, y n Represents the actual emotional feature label of the nth text. In an optional embodiment, the actual emotional feature label of the text can be represented by one-hot encoding.

[0151] Furthermore, the second classification model is used to calculate the probability of the corresponding sub-class emotion category, as shown in the following formula (10).

[0152]

[0153] Among them, p(c=xiaolei|T n ) represents the probability that the nth text belongs to the xiaoleith sub-category of emotion under the daleith major category of emotion, which is the output value of the second classification model. Then, the second classification loss is calculated using the sub-category emotion category results, as shown in the following formula (11).

[0154] L xiao =-y n log(p(c=xiaolei|T n )) (11)

[0155] After obtaining the first classification loss and the second classification loss, a target loss function is constructed according to the first classification loss and the second classification loss. Specifically, the process of constructing the target loss function includes:

[0156] Determining a first product of the first classification loss and a preset weight coefficient;

[0157] Determine a difference between 1 and the weight coefficient, and determine a second product of the second classification loss and the difference;

[0158] Based on the first product and the second product, a target loss function is constructed.

[0159] More specifically, the obtained target loss function is shown in the following formula (12).

[0160] L duoji =αL da +(1-α)L xiao (12)

[0161] Among them, α is the weight coefficient used to balance the first classification loss and the second classification loss. By optimizing the multi-level sentiment loss, it is possible to comprehensively consider the information of the parent class sentiment while performing fine-grained classification of sub-class sentiment. This can improve the ability of the sentiment classification model to distinguish different parent class sentiments and enable more accurate sentiment analysis of texts.

[0162] As a specific example, both the parent emotion category and the child emotion category use one-hot encoding to determine the actual emotion label of the text, as shown below:

[0163] Forward: [1, 0, 0];

[0164] Neutral: [0, 1, 0];

[0165] Negative: [0, 0, 1];

[0166] like: [1, 0, 0, 0, 0, 0, 0];

[0167] happiness: [0, 1, 0, 0, 0, 0, 0];

[0168] surprise: [0, 0, 1, 0, 0, 0, 0];

[0169] Angry: [0, 0, 0, 1, 0, 0, 0, 0];

[0170] fear: [0, 0, 0, 1, 0, 0, 0];

[0171] disgust: [0, 0, 0, 0, 1, 0, 0];

[0172] sad: [0, 0, 0, 0, 0, 1, 0];

[0173] Neutral: [0, 0, 0, 0, 0, 0, 1].

[0174] Assume that the probability of a sample belonging to positive text is 0.7, the probability of neutral text is 0.2, and the probability of negative text is 0.1. Therefore, the sentiment of the sample is positive, and the first classification loss of the sample is L zhengxiang =-[1, 0, 0]*log0.7=-1*log0.7=0.15.

[0175] Assume that the probabilities of the sample belonging to like, happiness, surprise, anger, fear, disgust, sadness, and neutral are 0.3, 0.2, 0.1, 0.2, 0.15, 0.02, and 0.03 respectively. Therefore, the emotion of the sample belongs to like, and the second classification loss of the sample is L xihuan =-[1, 0, 0, 0, 0, 0, 0, 0]*log0.3=-1*log0.3=0.52.

[0176] Assuming the balance weight coefficient is 0.6, the target loss of this sample is L duoji =0.6*0.15+0.4*0.52=0.67.

[0177] In order to describe the emotion of the text in more detail, the text emotion classification method of the embodiment of the present invention divides the text emotion into three parent emotion categories: positive, negative and neutral, and eight sub-emotion categories: like, happiness, surprise, anger, fear, disgust, sadness, and none (i.e., like, happiness, surprise, anger, fear, disgust, sadness, and neutral). In addition, a multi-level emotion classification loss function (i.e., target classification loss function) is constructed for the parent emotion information contained in the sub-emotion categories. Furthermore, in order to classify the text emotion more accurately, the embodiment of the present invention constructs a multi-level emotion classification model (i.e., a pre-trained emotion classification model) based on the multi-level emotion classification loss function.

[0178] The text sentiment classification method of the embodiment of the present invention determines the word features, phrase features and sentence features of the text to be classified, and integrates the word features, phrase features and sentence features to obtain the target features of the text to be classified; according to the target features of the text to be classified and the pre-trained sentiment classification model, the technical means of determining the sentiment of the text to be classified is used, so a large amount of work of manually annotating data is saved and the labor cost is reduced; based on the word features, phrase features and sentence features obtained by the analysis model of three granularities, the multi-level sentiment features about the text are obtained by integrating the three features, and then the sentiment of the text is determined by the multi-level sentiment classification model and the multi-level sentiment features, thereby improving the accuracy of text classification. Specifically, in order to more clearly describe the multi-level text sentiment, the relationship between the parent class sentiment is considered while performing the sub-class sentiment classification. The method of the embodiment of the present invention subdivides the text sentiment classification into two parts: the parent class sentiment classification and the sub-class sentiment classification, so as to achieve more accurate classification of the text.

[0179] Figure 7 is a schematic diagram of the main modules of the text sentiment classification device 700 according to an embodiment of the present invention. Figure 7 As shown, the device 700 includes:

[0180] The text acquisition module 701 is used to acquire the text to be classified;

[0181] A feature determination module 702 is used to determine the word features, phrase features and sentence features of the text to be classified, and to integrate the word features, phrase features and sentence features to obtain the target features of the text to be classified;

[0182] The sentiment determination module 703 is used to determine the sentiment of the text to be classified according to the target features of the text to be classified and the pre-trained sentiment classification model.

[0183] Optionally, the feature determination module 702 is further configured to:

[0184] Determining the word features of the text to be classified through a pre-trained word analysis model;

[0185] Determining the phrase features of the text to be classified by using a pre-trained phrase analysis model;

[0186] The sentence features of the text to be classified are determined by using a pre-trained sentence analysis model.

[0187] Optionally, the feature determination module 702 is further configured to:

[0188] Performing word segmentation processing on the text to be classified to obtain a plurality of words to be analyzed;

[0189] Determining word vectors of the multiple words to be analyzed;

[0190] Determining the word features of the text to be classified according to the word vector and the pre-trained word analysis model;

[0191] Determining phrase features of the text to be classified according to the word vector and the pre-trained phrase analysis model;

[0192] The sentence features of the text to be classified are determined based on the word vector and the pre-trained sentence analysis model.

[0193] Optionally, the pre-trained word analysis model includes a first convolutional layer and a first attention module.

[0194] Optionally, the feature determination module 702 is further used to: input the word vectors of any two consecutive words to be analyzed into the pre-trained phrase analysis model to determine the phrase features of the text to be classified.

[0195] Optionally, the pre-trained phrase analysis model includes a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a second attention module, a third attention module and a global maximum pooling layer.

[0196] Optionally, the feature determination module 702 is further configured to:

[0197] Input the word vectors of any two consecutive words to be analyzed into the fourth convolutional layer to obtain a first convolutional feature, and input the first convolutional feature into the second attention module to obtain a first phrase feature;

[0198] Inputting the first convolution feature into the fifth convolution layer to obtain a second convolution feature, and inputting the second convolution feature into the third attention module to obtain a second phrase feature;

[0199] Inputting the second convolution feature into the sixth convolution layer to obtain a third convolution feature, and inputting the third convolution layer feature into the global maximum pooling layer to obtain a third phrase feature;

[0200] The first phrase feature, the second phrase feature and the third phrase feature are integrated to obtain the phrase feature of the text to be classified.

[0201] Optionally, the pre-trained sentence analysis model includes a seventh convolutional layer and an LSTM layer.

[0202] Optionally, the feature determination module 702 is further configured to: before performing word segmentation processing on the text to be classified, perform denoising processing and stop word removal processing on the text to be classified according to preset rules.

[0203] Optionally, the device further comprises a model training module, which is used to:

[0204] Obtain training sample data;

[0205] Determine the parent emotion category and the first classification loss of the training sample data by using the pre-trained first classification model;

[0206] Determine the subclass sentiment category and the second classification loss of the training sample data by using the pre-trained second classification model;

[0207] Constructing a target loss function according to the first classification loss and the second classification loss;

[0208] The pre-trained sentiment classification model is trained according to the target loss function.

[0209] Optionally, the parent emotion category includes positive, negative and neutral; the child emotion category includes at least one or more of the following: like, happiness, surprise, anger, fear, disgust, sadness, and neutral.

[0210] Optionally, the model training module is also used to: determine a first product of the first classification loss and a preset weight coefficient; determine a difference between 1 and the weight coefficient, and determine a second product of the second classification loss and the difference; and construct a target loss function based on the first product and the second product.

[0211] The text sentiment classification device of the embodiment of the present invention determines the word features, phrase features and sentence features of the text to be classified, and integrates the word features, phrase features and sentence features to obtain the target features of the text to be classified; determines the technical means of the sentiment of the text to be classified according to the target features of the text to be classified and a pre-trained sentiment classification model, thereby saving a lot of work of manually annotating data and reducing labor costs; obtains multi-level sentiment features about the text by integrating the word features, phrase features and sentence features obtained by the analysis model based on three granularities, and then determines the sentiment of the text through a multi-level sentiment classification model and the multi-level sentiment features, thereby improving the accuracy of text classification.

[0212] The above device can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0213] Figure 8 An exemplary system architecture 800 is shown to which the text sentiment classification method or text sentiment classification device according to the embodiment of the present invention can be applied.

[0214] like Figure 8As shown, system architecture 800 may include terminal devices 801, 802, 803, a network 804, and a server 805. Network 804 is used to provide a medium for communication links between terminal devices 801, 802, 803 and server 805. Network 804 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0215] Users can use terminal devices 801, 802, and 803 to interact with server 805 through network 804 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 801, 802, and 803, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0216] The terminal devices 801 , 802 , and 803 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0217] The server 805 may be a server that provides various services, such as a backend management server that provides support for shopping websites browsed by users using the terminal devices 801, 802, and 803. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information and product information) to the terminal device.

[0218] It should be noted that the text sentiment classification method provided in the embodiment of the present invention is generally executed by the server 805 , and accordingly, the text sentiment classification device is generally arranged in the server 805 .

[0219] It should be understood that Figure 8 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0220] Reference below Fig. 9 , which shows a schematic diagram of the structure of a computer system 900 of a terminal device suitable for implementing an embodiment of the present invention. Fig. 9 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0221] like Fig. 9As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 909 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the system 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0222] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

[0223] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present invention are executed.

[0224] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0225] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0226] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be set in a processor. For example, they may be described as: a processor includes a sending module, an acquisition module, a determination module, and a first processing module. The names of these modules do not, in some cases, constitute a limitation on the unit itself. For example, the sending module may also be described as a "module for sending a picture acquisition request to the connected server."

[0227] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes:

[0228] Get the text to be classified;

[0229] Determine the word features, phrase features and sentence features of the text to be classified, and fuse the word features, phrase features and sentence features to obtain the target features of the text to be classified;

[0230] The sentiment of the text to be classified is determined according to the target features of the text to be classified and a pre-trained sentiment classification model.

[0231] The technical solution of the embodiment of the present invention saves a lot of work on manual data annotation and reduces labor costs; based on the word features, phrase features and sentence features obtained by the analysis model of three granularities, multi-level sentiment features about the text are obtained by fusing the three features, and then the sentiment of the text is determined through a multi-level sentiment classification model and the multi-level sentiment features, thereby improving the accuracy of text classification.

[0232] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A text sentiment classification method, characterized in that: include: Get the text to be classified; Determine the word features, phrase features and sentence features of the text to be classified, and fuse the word features, phrase features and sentence features to obtain the target features of the text to be classified; Determine the sentiment of the text to be classified according to the target features of the text to be classified and a pre-trained sentiment classification model; wherein the pre-trained sentiment classification model is trained according to the following process: obtain training sample data; determine the parent class sentiment category and the first classification loss of the training sample data using a pre-trained first classification model; determine the subclass sentiment category and the second classification loss of the training sample data using a pre-trained second classification model; construct a target loss function according to the first classification loss and the second classification loss; and train the pre-trained sentiment classification model according to the target loss function.

2. The method according to claim 1, characterized in that Determining the word features, phrase features and sentence features of the text to be classified includes: Determining the word features of the text to be classified through a pre-trained word analysis model; Determining the phrase features of the text to be classified by using a pre-trained phrase analysis model; The sentence features of the text to be classified are determined by using a pre-trained sentence analysis model.

3. The method according to claim 2, characterized in that Determining the word features, phrase features and sentence features of the text to be classified includes: Performing word segmentation processing on the text to be classified to obtain a plurality of words to be analyzed; Determining word vectors of the multiple words to be analyzed; Determining the word features of the text to be classified according to the word vector and the pre-trained word analysis model; Determining phrase features of the text to be classified according to the word vector and the pre-trained phrase analysis model; The sentence features of the text to be classified are determined based on the word vector and the pre-trained sentence analysis model.

4. The method according to claim 3, characterized in that The pre-trained word analysis model includes a first convolutional layer and a first attention module.

5. The method according to claim 3, characterized in that: Determining the phrase features of the text to be classified according to the word vector and the pre-trained phrase analysis model includes: The word vectors of any two consecutive words to be analyzed are input into the pre-trained phrase analysis model to determine the phrase features of the text to be classified.

6. The method according to claim 5, characterized in that The pre-trained phrase analysis model includes a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a second attention module, a third attention module and a global maximum pooling layer.

7. The method according to claim 6, characterized in that Determining the phrase features of the text to be classified according to the word vector and the pre-trained phrase analysis model includes: Input the word vectors of any two consecutive words to be analyzed into the fourth convolutional layer to obtain a first convolutional feature, and input the first convolutional feature into the second attention module to obtain a first phrase feature; Inputting the first convolution feature into the fifth convolution layer to obtain a second convolution feature, and inputting the second convolution feature into the third attention module to obtain a second phrase feature; Inputting the second convolutional feature into the sixth convolutional layer to obtain a third convolutional feature, and inputting the third convolutional feature into the global maximum pooling layer to obtain a third phrase feature; The first phrase feature, the second phrase feature and the third phrase feature are integrated to obtain the phrase feature of the text to be classified.

8. The method according to claim 3, characterized in that The pre-trained sentence analysis model includes a seventh convolutional layer and an LSTM layer.

9. The method according to claim 3, characterized in that: Before performing word segmentation processing on the text to be classified, the method further includes: performing denoising processing and stop word removal processing on the text to be classified according to preset rules.

10. The method according to claim 1, characterized in that The parent emotion categories include positive, negative and neutral; the child emotion categories include at least one or more of the following: like, happiness, surprise, anger, fear, disgust, sadness, and neutral.

11. The method according to claim 1, characterized in that: According to the first classification loss and the second classification loss, constructing the target loss function includes: Determining a first product of the first classification loss and a preset weight coefficient; Determine a difference between 1 and the weight coefficient, and determine a second product of the second classification loss and the difference; Based on the first product and the second product, a target loss function is constructed.

12. A text sentiment classification device, characterized in that: include: A text acquisition module is used to acquire the text to be classified; A feature determination module, used to determine the word features, phrase features and sentence features of the text to be classified, and to fuse the word features, phrase features and sentence features to obtain the target features of the text to be classified; The emotion determination module is used to determine the emotion of the text to be classified according to the target features of the text to be classified and a pre-trained emotion classification model; wherein the pre-trained emotion classification model is trained according to the following process: obtaining training sample data; using the pre-trained first classification model, determining the parent class emotion category and the first classification loss of the training sample data; using the pre-trained second classification model, determining the subclass emotion category and the second classification loss of the training sample data; constructing a target loss function according to the first classification loss and the second classification loss; and training the pre-trained emotion classification model according to the target loss function.

13. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 11.

14. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images