Context-dependent multi-classification sentiment analysis method and system based on CNN-BiLSTM framework

Abstract

The present invention provides a context-dependent multi-classification sentiment analysis method and system based on a CNN-BiLSTM framework, and relates to the technical field of text information processing. The present invention obtains text comments and pre-processes the text comments; performs multiple sentiment category labeling processing on clauses of the text comments to obtain a training set, a validation set, and a test set; obtains a multi-classification sentiment analysis model based on a context-dependent strategy, a training set, a validation set, and a test set and a sentiment analysis model based on a CNN-BiLSTM framework, and analyzes the sentiment category of the text comment to be tested through the multi-classification sentiment analysis model. The present invention proposes a multi-classification sentiment analysis model, which is constructed based on a context-dependent strategy, captures more complex sentiment features by fusing the sentiment information of the context, can more effectively identify the sentiment of each clause in the comment, improves the accuracy of the multi-classification sentiment analysis model, and can more accurately analyze the sentiment category of the text comment to be tested.

Description

Technical Field

[0001] The present invention relates to the technical field of text information processing, and in particular to a context-dependent multi-classification sentiment analysis method and system based on a CNN-BiLSTM framework. Background Art

[0002] With the rapid development of social networks, more and more users have posted text reviews about various products on forums, blogs and other websites. Through effective analysis and mining, text reviews can help companies capture customer needs in real time and guide consumers' shopping decisions. Sentiment analysis, as an important text mining technology, has received widespread attention recently.

[0003] The existing recognition of multiple emotions is called multi-class sentiment analysis or multi-class sentiment classification. In recent years, some studies on multi-class sentiment analysis have been conducted. These studies mainly use machine learning methods such as support vector machine (SVM) and random forest to recognize multiple classes of emotions.

[0004] However, the inventors of the present application have found that the existing multi-category sentiment classification has certain limitations. First, the specific meaning of sentiment in multi-category sentiment analysis has not been discussed in depth. Many studies have only discussed three-category sentiment analysis, including positive, negative, and neutral sentiment, or sentiment intensity analysis based on a scale from 1 to 5, where 1 represents very negative sentiment and 5 represents very positive sentiment. Secondly, the granularity of multi-category sentiment analysis is insufficient. In most studies on multi-category sentiment analysis, the sentiment of a text comment is either identified as a single sentiment type, or a text comment is labeled with multiple sentiment tags. The specific sentiment expressed by each clause may be different, so it cannot be accurately identified, resulting in the low accuracy of existing multi-category sentiment analysis methods. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the shortcomings of the prior art, the present invention provides a context-dependent multi-classification sentiment analysis method and system based on a CNN-BiLSTM framework, which solves the technical problem of low accuracy of the existing multi-classification sentiment analysis methods.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] The present invention provides a context-dependent multi-classification sentiment analysis method based on a CNN-BiLSTM framework, comprising:

[0010] Acquire text comments, preprocess the text comments, and obtain clauses of the text comments;

[0011] Perform multiple sentiment category labeling on the clauses of the text reviews to obtain training sets, validation sets, and test sets;

[0012] A multi-classification sentiment analysis model is obtained based on a context-dependent strategy, the training set, the validation set and the test set, and a sentiment analysis model based on a CNN-BiLSTM framework. The sentiment categories of the text comments to be tested are analyzed by the multi-classification sentiment analysis model. The structure of the multi-classification sentiment analysis model includes a parsing layer, an embedding layer, a convolutional layer, a pooling layer, a bidirectional LSTM layer and a Softmax layer. The bidirectional LSTM layer is used to learn text context relevance based on a context-dependent strategy.

[0013] Preferably, the pretreatment comprises:

[0014] Delete spam comments;

[0015] Perform word segmentation on text comments;

[0016] Perform clause cutting on the comment text and divide the text comment r into T clauses clause1, clause2,…, clause t ,…,clause T .

[0017] Preferably, the context-dependent strategy includes:

[0018] The sentiment category of each clause in the comment text is obtained by transferring the sentiment between clauses in the comment text. The expression is as follows:

[0019]

[0020] Among them, clause k Indicates the kth clause, tag k Indicates clause i The sentiment category, F2 represents the feature extraction function of the clause relationship, represents the weight of the features of the i-th adjacent clause on the k-th clause.

[0021] Preferably, the parsing layer includes:

[0022] The text review data is divided into clauses based on grammatical parsing.

[0023] Preferably, the embedding layer is used to convert text comments into vector data, specifically:

[0024] Use the trained word vector model to vectorize each word. By sequentially linking the word vectors, the matrix of each clause can be obtained. in, Indicates M t Clause t where d is the dimension of the word vector and N is the maximum length of the clause.

[0025] The training process of the word vector model is as follows:

[0026] We chose to use the gensim package in Python to train the word vector model. By repeatedly training a large amount of text review corpus, we obtained a word vector model for a specific field.

[0027] Preferably, the convolutional layer is used to extract n-gram grammatical features from the matrix of each clause generated by the embedding layer, and the n-gram grammatical features represent the sentiment information of each clause, specifically:

[0028] Three convolution kernels with different heights, the same width and the same convolution kernel are used to extract n-gram grammatical features from the matrix of each clause generated from the embedding layer, through the matrix M t With filter F l Continuous convolution operations between (1≤l≤3L) can obtain n-gram feature maps Where ω∈{2,3,4} represents the filter F l The height, a characteristic is obtained from δ words x by the following formula n:n+δ-1 The following window is generated:

[0029]

[0030] in, is the convolution operator, W l and b l They represent the weight matrix and the bias of the lth filter respectively, and f is the ReLU function.

[0031] Preferably, the pooling layer is used to sequentially stack the L feature maps generated by the three types of convolution kernels in the convolution layer into a matrix As shown below:

[0032]

[0033] Among them, ω P Represents the height of the P-th type of convolution kernel. Then, take the maximum value of each column to get the new feature

[0034]

[0035] Through the maximum pooling operation, the final feature vector associated with the P-th convolution kernel can be expressed as:

[0036]

[0037] Finally, the final feature vector of the t-th clause is generated by concatenating the three feature maps associated with different types of convolutional kernels:

[0038]

[0039] Preferably, the LSTM layer is used to capture the long-distance dependencies in the clauses. The LSTM network introduces a storage unit c t And three gates to process the input vector. These three gates can remember important information over multiple time steps. The precise definition of LSTM is as follows:

[0040] f t =σ(W f ·[h t-1 ,g t ]+b f )

[0041] i t =σ(W i ·[h t-1 ,g t ]+b i )

[0042] o t =σ(W o ·[h t-1 ,g t ]+b o )

[0043] c t =f t ⊙c t-1 +i t ⊙tanh(W c ·[h t-1 ,g t ]+b c )

[0044] h t =o t ⊙tanh(c t )

[0045] Among them, W f , W i , W o , W c and b f , b i , b o, b c They represent the weight matrix and bias in the model training process, ⊙ represents element-by-element multiplication, and h t represents the hidden state and σ is the Sigmoid function. Considering that LSTM can only capture the sentiment information of the left clause, the bidirectional LSTM is applied to capture the sentiment information of the left and right clauses. Bi-LSTM uses two independent LSTM units, one for forward and one for reverse. The final hidden state of the Bi-LSTM model is From the hidden state of the forward LSTM and the hidden state of the reverse LSTM Connection composition:

[0046]

[0047] Hidden State is a clause t The feature vector that integrates the sentiment information across clauses. For ease of understanding, it is represented as g t ',

[0048] Preferably, the Softmax layer is used to predict the clause t Probability distribution over sentiment labels

[0049]

[0050] Among them, W q and b q represent the weight matrix and bias respectively, while Y represents the number of sentiment labels. The label with the highest probability value is the clause that the model finally judges. t The label formula is as follows:

[0051]

[0052] The labels of all clauses segmented by the text review r can be expressed as:

[0053] tag r =(tag1,tag2,…,tag t ,…,tag T ).

[0054] The present invention also provides a context-dependent multi-classification sentiment analysis system based on a CNN-BiLSTM framework, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-described method when executing the computer program.

[0055] (III) Beneficial effects

[0056] The present invention provides a context-dependent multi-classification sentiment analysis method and system based on the CNN-BiLSTM framework. Compared with the prior art, it has the following beneficial effects:

[0057] The present invention pre-processes the text comments by acquiring text comments; performs multiple emotion category annotation processing on the clauses of the text comments to acquire a training set, a validation set and a test set; acquires a multi-classification emotion analysis model based on a context-dependent strategy, a training set, a validation set and a test set and an emotion analysis model based on a CNN-BiLSTM framework, analyzes the emotion category of the text comments to be tested by the multi-classification emotion analysis model, and the structure of the multi-classification emotion analysis model includes a parsing layer, an embedding layer, a convolution layer, a pooling layer, a bidirectional LSTM layer and a Softmax layer. The bidirectional LSTM layer is used to learn the text context relevance based on a context-dependent strategy. The present invention proposes a multi-classification emotion analysis model, which is constructed based on a context-dependent strategy, captures more complex emotion features by fusing the emotion information of the context, can more effectively identify the emotion of each clause in the comment, improves the accuracy of the multi-classification emotion analysis model, and can more accurately analyze the emotion category of the text comments to be tested. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 A block diagram of a context-dependent multi-classification sentiment analysis method based on a CNN-BiLSTM framework according to an embodiment of the present invention;

[0060] Figure 2 A schematic diagram of a grammatical parsing tree of a comment in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] The embodiment of the present application solves the technical problem of low accuracy of existing multi-category sentiment analysis methods by providing a context-dependent multi-classification sentiment analysis method based on a CNN-BiLSTM framework, and achieves accurate analysis of the sentiment category of the text comments to be tested.

[0063] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0064] With the rapid development of social media, mining customer opinions based on text reviews is crucial to understanding consumer needs. The emotions expressed by customers in product reviews are often diverse, such as worry, satisfaction, pity, tolerance, etc. However, the accuracy of existing multi-category sentiment analysis methods is low, resulting in a certain deviation between the derived sentiment information and the actual situation. In an embodiment of the present invention, a context-dependent multi-classification sentiment analysis method using a CNN-BiLSTM framework is proposed. This method can effectively identify the sentiment of each clause in a product review by fusing contextual sentiment information, and can more accurately analyze the sentiment category of the text review to be tested.

[0065] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0066] The embodiment of the present invention provides a context-dependent multi-classification sentiment analysis method based on a CNN-BiLSTM framework, the method being executed by a computer, such as Figure 1 As shown, it includes steps S1 to S3:

[0067] S1. Obtain text comments, preprocess the text comments, and obtain clauses of the text comments;

[0068] S2. Label the clauses of the text comments with multiple sentiment categories to obtain training sets, validation sets, and test sets;

[0069] S3. A multi-classification sentiment analysis model is obtained based on a training set, a validation set, a test set and a sentiment analysis model based on a CNN-BiLSTM (convolutional neural network-bi-directional long short-term memory) framework. The sentiment categories of the text comments to be tested are analyzed by the multi-classification sentiment analysis model. The structure of the multi-classification sentiment analysis model includes a parsing layer, an embedding layer, a convolutional layer, a pooling layer, a bidirectional LSTM layer and a Softmax layer. The bidirectional LSTM layer is used to learn text context relevance.

[0070] The embodiment of the present invention proposes a multi-classification sentiment analysis model, which is constructed based on a context-dependent strategy. By fusing the sentiment information of the context to capture more complex sentiment features, it can more effectively identify the sentiment of each clause in the comment, thereby improving the accuracy of the multi-classification sentiment analysis model, thereby more accurately analyzing the sentiment category of the text comment to be tested.

[0071] In one embodiment, S1, obtain text comments, pre-process the text comments, and obtain clauses of the text comments. The specific implementation process is as follows:

[0072] S101. Capture text comment data. Based on the Scrapy framework developed in Python, we extract and parse web page data through XPath and CSS expressions, use the Redis database as a distributed shared crawler queue, and the MongoDB database as a data repository. We integrate the Selenium automated testing tool, use random User-Agent, Alibaba Cloud proxy IP, and self-built proxy IP pools, and deploy to cloud servers to achieve large-scale real-time incremental crawling of product text comment data on multiple social media platforms.

[0073] S102, data preprocessing. Further process the crawled data to obtain high-quality and effective data. Specifically include:

[0074] S102a, spam text comment removal. In order to improve the quality of data, spam text comments need to be removed. Spam text comment removal mainly includes the following two points:

[0075] 1) Eliminate by keywords: Eliminate spam text comments by summarizing some keywords such as "html", "phone", "delete", etc.

[0076] 2) Eliminate by length of text comments. Text comments that are too long or too short may be spam text comments. Therefore, the length of text comments is limited to between 5 and 150, and text comments of other lengths are eliminated.

[0077] S102b, text segmentation. The embodiment of the present invention uses the Chinese word segmentation system ICTCLAS, which is called by Python, to implement the word segmentation of text comments.

[0078] S102c, clause segmentation. In the embodiment of the present invention, a clause is the basic unit of analysis. Therefore, a text comment needs to be segmented into a set of clauses for further analysis. The embodiment of the present invention uses the Stanford parser to perform grammatical analysis on each text, and divides the text comment r into T clauses, namely, clause1, clause2, ..., clause t ,…,clauseT Each text comment is parsed into a tree graph, where clauses are labeled as IP or CP. However, there may be embedded relationships between clauses. Therefore, the parse tree is recursively traversed to extract leaf clauses that do not contain other IP or CP nodes in their child nodes as basic units. The grammatical parse tree of a text comment is as follows: Figure 2 shown.

[0079] In one embodiment, S2, multiple sentiment category labeling is performed on the clauses of the text review to obtain a training set, a validation set, and a test set. The specific implementation process is as follows:

[0080] Since the text review data extracted from the web page is unlabeled, manual annotation is required to obtain the training set, validation set, and test set that the model can use. First, the emotions that may be expressed in the product text review data are stipulated as the following nine types: satisfaction, trust, happiness, tolerance, neutrality, pity, worry, disappointment, and anger. The first four are positive emotions, and the last four are negative emotions. The specific descriptions are shown in Table 1. Then, select some text review data for manual annotation, and annotate each clause of these text reviews as one of the above nine emotions. Finally, after the data is annotated, these product text reviews are divided into training set, validation set, and test set according to 6:2:2.

[0081] Table 1. Detailed description of the nine possible emotions expressed in product text reviews

[0082]

[0083]

[0084] In one embodiment, S3, a multi-classification sentiment analysis model is obtained based on a context-dependent strategy, a training set, a validation set, a test set, and a sentiment analysis model based on a CNN-BiLSTM framework, and the sentiment categories of the text comments to be tested are analyzed by the multi-classification sentiment analysis model.

[0085] It should be noted that, in the embodiment of the present invention, the context-dependent strategy refers to the emotional transmission between clauses. Usually, each clause constituting a comment expresses only one specific emotion. For example, the text comment R1 represents a product text review about a car, which consists of three clauses R1a-R1c, which express the emotions of satisfaction, pity, and disappointment, respectively. Specifically, the text reviewer first expressed satisfaction with the space of the car, and then used the conjunction "but", and the emotion turned into pity for the interior of the car. Finally, through the in-depth description of the product features, including the materials and workmanship as sub-features of the interior, the text reviewer's emotion changed from pity to disappointment.

[0086] (R1) This car has a lot of space, not crowded at all, (R1.a) But the regret is the interior of the car, (R1.b) both the materials and the workmanship are very disappointing. R1.c)

[0087] Although clauses may express similar or even opposite emotions, the emotions they express are usually not independent. Usually, they have a transitive relationship. In fact, users often clearly express their opinions on certain products by using short clauses to express emotions in text reviews. In addition, users will subconsciously make these clauses coherent at the lexical level or semantic level. In other words, the emotions of these clauses constitute a process of continuous emotion expression. For example, in text review R1, R1a–R1c express different emotions, but their combination reflects the overall impression of the text reviewer on the product, providing a continuous emotion expression.

[0088] In this paper, the phenomenon of continuous emotional expression between text review clauses is called emotional transfer. Based on a large number of observations of product text reviews, it is found that emotional transfer is ubiquitous. Generally, there are three main ways of emotional transfer: transition, deepening and stabilization.

[0089] 1) Transition. Sentiment transition refers to the phenomenon that adjacent clauses express opposite sentiments. For example, in the text review R2, clause R2.a expresses positive sentiment, while clause R2.b expresses negative sentiment.

[0090] (R2) Compared with other aspects, this car is pretty good. (R2.a) But the sound insulation is not very good and the noise is a bit loud. (R2.c)

[0091] 2) Deepening. The deepening of emotion refers to the phenomenon that a clause expresses stronger emotion than the previous clause. For example, in the text review R3, R3.d contains the word “especially” to emphasize the emotion of satisfaction.

[0092] (R3) I have used my B30 for almost half a year. So far, I have not had any of the minor problems others have mentioned. (R3.a) It has plenty of power. (R3.b) The space is also quite satisfactory. (R3.c) It is especially worth mentioning that the cruise control is quite useful. I rely on it when driving on the highway. (R3.d)

[0093] 3) Stability. The stability of sentiment refers to the phenomenon that adjacent clauses express similar sentiments. For example, all clauses in the text review R4 express satisfied sentiments, which are relatively stable without large fluctuations.

[0094] (R4) This car is very stable to drive, (R4.a) and has good handling, (R4.b) so far, my car has not lost much value, (R4.c) I am very pleased. (R4.d)

[0095] Under this description, emotional transfer can be defined as follows:

[0096] Emotional transfer refers to the phenomenon that the clauses of a text comment express continuous emotions in different ways such as transition, deepening and stabilization.

[0097] Based on the sentiment transfer defined above, lexical or semantic connections between clauses can be established, and the sentiment information of all clauses of a text review can be fused. In this way, richer features of the clauses can be extracted, and the sentiment of the clauses can also be analyzed more accurately.

[0098] Assuming that a text review consists of T clauses, without considering sentiment transmission, the sentiment of each clause can be calculated as follows:

[0099] tag k =Model(F1[clause k ])

[0100] Among them, clause k represents the kth clause, F1 represents the feature extraction function of the clause sentiment information, tag k Indicates clause i In this case, the features are only extracted from the current clause, and the representation ability is relatively limited.

[0101] On the other hand, considering sentiment transfer, the sentiment of each clause can be calculated as follows:

[0102]

[0103] Among them, F2 represents the feature extraction function of the clause relationship, It represents the weight of the feature of the i-th adjacent clause to the k-th clause. From the above formula, we can see that F1 F2 is used to represent the existing feature extraction function. The use of these two representative functions shows that there are more ways to extract features, richer features can be extracted and the representation ability has been improved.

[0104] The specific implementation process of step S3 is as follows:

[0105] S301, using the training set obtained in step S2 to train the sentiment analysis model based on the CNN-BiLSTM framework to obtain a multi-classification sentiment analysis model.

[0106] S302: Optimize the multi-classification sentiment analysis model using the validation set and the test set to improve the generalization ability of the model. Specifically:

[0107] By minimizing the true probability distribution y and the predicted probability distribution The cross entropy error between is used to optimize the multi-class sentiment analysis model. Considering the problem of sample imbalance in multi-classification tasks, weights are set for each category:

[0108]

[0109] Among them, amount j represents the number of samples in the jth category. Given a training set with X training samples, the loss function can be expressed as:

[0110]

[0111] Among them, T i Indicates the number of clauses divided by the i-th sample.

[0112] S303: Analyze the sentiment category of the text review to be tested through the optimized multi-classification sentiment analysis model. Specifically:

[0113] The text comments to be tested are input into the multi-classification sentiment analysis model, which divides the text comments to be tested into a set of clauses, and then identifies the sentiment category of each clause. Enterprises can summarize the sentiment category results of a large amount of text comment data through the above method to obtain the number of each sentiment category, thereby quantifying the opinions of consumers and better improving and innovating products based on these opinion information. It should be noted that once the multi-classification sentiment analysis model is optimized, the sentiment category of the text comments to be tested can be analyzed repeatedly without repeated training, that is, steps S1 to S302 are executed once, and step S303 can be executed multiple times. At the same time, before the text comments to be tested are input into the multi-classification sentiment analysis model, they only need to perform the preprocessing of removing junk text comments, and the steps such as text segmentation and clause cutting are automatically performed by the classification sentiment analysis model.

[0114] In the embodiment of the present invention, the proposed multi-classification sentiment analysis model includes a parsing layer, an embedding layer, a convolutional layer, a pooling layer, a bidirectional LSTM layer and a Softmax layer. First, the multi-classification sentiment analysis model divides the given text comment r into T clauses based on grammatical parsing, namely, clause1, clause2,…, clause t ,…,clause T . Then, in the embedding layer, each word is vectorized by the word vector model. Subsequently, once the word vectors pass through the convolutional layer and the maximum pooling layer sequentially, the n-gram features of each clause can be extracted. Then, the context relevance is learned based on the bidirectional LSTM, and the local feature vectors are sequentially integrated in the clauses to construct a new feature vector for each clause. Finally, once the feature vector of each clause passes through the softmax layer, its sentiment prediction can be obtained. The various layers of the multi-classification sentiment analysis model are described in detail below.

[0115] 1) Parsing layer. The parsing layer divides the text comment data into clauses based on grammatical analysis. The process is similar to step S102c.

[0116] 2) Embedding layer. The embedding layer converts text comments into vector data. First, each word is vectorized using the trained word vector model. Then, by sequentially linking the word vectors, the matrix of each clause can be obtained. Among them, it means M t Clause t where d is the dimension of the word vector and N is the maximum length of a clause. If the length of a clause is less than N, some random vectors with normal distribution μ(-1 / d,1 / d) are filled in.

[0117] The training process of the word vector model is as follows:

[0118] The function of the word vector model is to assign an independent vector to each word in the text, so that the text can be vectorized. This vectorization is the basic expression of text in the deep learning-based model. In this embodiment of the present invention, the gensim package in Python is used to train the word vector model. By repeatedly iteratively training a large number of product text review corpora, a word vector model for a specific field is obtained.

[0119] 3) Convolutional layer. The convolutional layer is used to extract n-gram grammatical features from the matrix of each clause generated by the embedding layer. In order to extract rich features, the embodiment of the present invention applies three types of convolution kernels, which have different heights (2, 3 and 4) and the same width d. Each type of convolution kernel has L filters. Through the matrix M t With filter F l Continuous convolution operations between (1≤l≤3L) can obtain n-gram feature maps Where ω∈{2,3,4} represents the filter F l Specifically, a feature is obtained from δ words x in the following way n:n+δ-1 The following window is generated:

[0120]

[0121] in, is the convolution operator, W l and b l They represent the weight matrix and the bias of the lth filter respectively, and f is the ReLU function.

[0122] 4) Pooling layer. In order to simplify the network complexity and retain the most important information, the maximum pooling layer is used to subsample the output of the convolution layer. In the multi-class sentiment analysis model of the embodiment of the present invention, max-1 merging is performed on the feature maps generated by each type of convolution kernel. Specifically, the L feature maps generated by each type of convolution kernel are first stacked sequentially into a matrix As shown below:

[0123]

[0124] Among them, ω P Represents the height of the P-th type of convolution kernel. Then, take the maximum value of each column to get the new feature

[0125]

[0126] Through the maximum pooling operation, the final feature vector associated with the P-th convolution kernel can be expressed as:

[0127]

[0128] Finally, the final feature vector of the t-th clause is generated by concatenating the three feature maps associated with different types of convolutional kernels.

[0129]

[0130] 5) LSTM layer. In order to learn the sentiment information in the clause into the feature vector of each clause, the LSTM layer learns the text context relevance based on the context dependency strategy. The feature vectors of each clause generated in the previous step are sequentially integrated and transmitted to the LSTM network. The LSTM network introduces a storage unit c t and three gates (i.e., forget gate f t , input gate i t and output gate o t ) to process the input vector, these three gates can remember important information over multiple time steps. The precise definition of LSTM is as follows:

[0131] f t =σ(W f ·[h t-1 ,g t ]+b f )

[0132] i t =σ(W i ·[h t-1 ,g t ]+b i )

[0133] ot =σ(W o ·[h t-1 ,g t ]+b o )

[0134] c t =f t ⊙c t-1 +i t ⊙tanh(W c ·[h t-1 ,g t ]+b c )

[0135] h t =o t ⊙tanh(c t )

[0136] Among them, W f , W i , W o , W c and b f , b i , b o , b c They represent the weight matrix and bias in the model training process, ⊙ represents element-by-element multiplication, and h t represents the hidden state and σ is the Sigmoid function. In addition, considering that LSTM can only capture the sentiment information of the left clause, a bidirectional LSTM is applied to capture the sentiment information of both the left and right clauses. Bi-LSTM uses two independent LSTM units, one for the forward direction and one for the reverse direction. The final hidden state of the Bi-LSTM model is From the hidden state of the forward LSTM and the hidden state of the reverse LSTM Connection composition.

[0137]

[0138] Hidden State is a clause t The feature vector that integrates the sentiment information across clauses. For ease of understanding, it is represented as g t '.

[0139]

[0140] 6) Softmax layer. After the LSTM layer, a softmax layer is used to predict the clause t Probability distribution over sentiment labels

[0141]

[0142] Among them, W q and b q Denote the weight matrix and bias respectively, while Y denotes the number of sentiment labels. The label with the highest probability value is the clause that the model finally judges. t The formula is as follows:

[0143]

[0144] The labels of all clauses segmented by the text review r can be expressed as:

[0145] tag r =(tag1,tag2,…,tag t ,…,tag T )

[0146] An embodiment of the present invention also provides a context-dependent multi-classification sentiment analysis system based on a CNN-BiLSTM framework, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the computer program.

[0147] It can be understood that the above-mentioned context-dependent multi-classification sentiment analysis system based on the CNN-BiLSTM framework provided in the embodiment of the present invention corresponds to the above-mentioned context-dependent multi-classification sentiment analysis method based on the CNN-BiLSTM framework. The explanations, examples, beneficial effects and other parts of the relevant contents can refer to the corresponding contents in the context-dependent multi-classification sentiment analysis method based on the CNN-BiLSTM framework, which will not be repeated here.

[0148] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0149] 1. An embodiment of the present invention proposes a multi-classification sentiment analysis model, which is constructed based on a context-dependent strategy. By integrating the sentiment information of the context to capture more complex sentiment features, it can more effectively identify the sentiment of each clause in the comment, thereby improving the accuracy of the multi-classification sentiment analysis model, thereby more accurately analyzing the sentiment category of the text comment to be tested.

[0150] 2. The embodiments of the present invention can analyze the sentiment of a sentence at the clause level to obtain more fine-grained sentiment information. By mining this fine-grained sentiment, enterprises can obtain more useful information, thereby better improving and innovating products.

[0151] It should be noted that, through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0152] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0153] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A context-dependent multi-classification sentiment analysis method based on CNN-BiLSTM framework, characterized in that: include: Acquire text comments, preprocess the text comments, and obtain clauses of the text comments; Perform multiple sentiment category labeling on the clauses of the text reviews to obtain training sets, validation sets, and test sets; A multi-classification sentiment analysis model is obtained based on the context dependency strategy, the training set, the validation set, the test set, and the sentiment analysis model based on the CNN-BiLSTM framework, and the sentiment category of the text review to be tested is analyzed by the multi-classification sentiment analysis model. The structure of the multi-classification sentiment analysis model includes a parsing layer, an embedding layer, a convolutional layer, a pooling layer, a bidirectional LSTM layer, and a Softmax layer. The bidirectional LSTM layer is used to learn the context relevance of the text based on the context dependency strategy; Wherein, the parsing layer is used to segment the text comment data into clauses based on grammatical parsing; The embedding layer is used to convert text comments into vector data, specifically: Use the trained word vector model to vectorize each word. By sequentially linking the word vectors, the matrix of each clause can be obtained. in, Indicates M t Clause t , d is the dimension of the word vector, and N is the maximum length of the clause; The training process of the word vector model is as follows: We chose to use the gensim package in Python to train the word vector model. By repeatedly training a large amount of text review corpus, we obtained a word vector model for a specific field. The convolutional layer is used to extract n-gram grammatical features from the matrix of each clause generated by the embedding layer. The n-gram grammatical features represent the sentiment information of each clause, specifically: Three convolution kernels with different heights, the same width and the same convolution kernel are used to extract n-gram grammatical features from the matrix of each clause generated from the embedding layer, through the matrix M t With filter F l Continuous convolution operations between (1≤l≤3L) can obtain n-gram feature maps Where ω∈{2,3,4} represents the filter F l The height, a characteristic is obtained from δ words x by the following formula n:n+δ-1 The following window is generated: in, is the convolution operator, W l and b l Represent the weight matrix and the deviation of the lth filter respectively, and f is the ReLU function; The pooling layer is used to sequentially stack the L feature maps generated by the three types of convolution kernels in the convolution layer into a matrix As shown below: Among them, ω P Represents the height of the P-th type of convolution kernel; take the maximum value of each column to get the new feature Through the maximum pooling operation, the final feature vector associated with the P-th convolution kernel can be expressed as: The final feature vector of the t-th clause is generated by concatenating the three feature maps associated with different types of convolutional kernels:

2. The context-dependent multi-classification sentiment analysis method based on the CNN-BiLSTM framework as claimed in claim 1, characterized in that: The pre-processing comprises: Delete spam comments; Perform word segmentation on text comments; Perform clause cutting on the comment text and divide the text comment r into T clauses clause1, clause2,..., clause t ,…,clause T .

3. The context-dependent multi-classification sentiment analysis method based on the CNN-BiLSTM framework as claimed in claim 2, characterized in that: The context-dependent strategies include: The sentiment category of each clause in the comment text is obtained by transferring the sentiment between clauses in the comment text. The expression is as follows: Among them, clause k Indicates the kth clause, tag k Indicates clause i The sentiment category, F2 represents the feature extraction function of the clause relationship, represents the weight of the features of the i-th adjacent clause on the k-th clause.

4. The context-dependent multi-classification sentiment analysis method based on the CNN-BiLSTM framework as claimed in claim 1, characterized in that The LSTM layer is used to capture the long-distance dependencies in the clauses. The LSTM network introduces a storage unit c t And three gates to process the input vector. These three gates can remember important information over multiple time steps. The precise definition of LSTM is as follows: f t =σ(W f ·[h t-1 ,g t ]+b f ) I t =σ(W i ·[h t-1 ,g t ]+b i ) the t =σ(W o ·[h t-1 ,g t ]+b o ) c t =f t ⊙c t-1 +i t ⊙tanh(W c ·[h t-1 ,g t ]+b c ) h t =o t ⊙tanh(c t ) Among them, W f , W i , W o , W c and b f , b i , b o , b c They represent the weight matrix and bias in the model training process, ⊙ represents element-by-element multiplication, and h t represents the hidden state and σ is the Sigmoid function. Considering that LSTM can only capture the sentiment information of the left clause, the bidirectional LSTM is applied to capture the sentiment information of the left and right clauses. Bi-LSTM uses two independent LSTM units, one for forward and one for reverse. The final hidden state of the Bi-LSTM model is From the hidden state of the forward LSTM and the hidden state of the reverse LSTM Connection composition: Hidden State is a clause t The feature vector that integrates the sentiment information across clauses is represented as g for ease of understanding. t ', 5. The context-dependent multi-classification sentiment analysis method based on the CNN-BiLSTM framework as claimed in claim 4, characterized in that: The Softmax layer is used to predict the clause t Probability distribution over sentiment labels Among them, W q and b q represent the weight matrix and bias respectively, while Y represents the number of sentiment labels. The label with the highest probability value is the clause that the model finally judges. t The label formula is as follows: The labels of all clauses segmented by the text review r can be expressed as: day r =(day1,day2,…,day t ,…,day T )。 6. A context-dependent multi-classification sentiment analysis system based on a CNN-BiLSTM framework, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

Images