A text sentiment classification method and system

Abstract

The present invention discloses a text sentiment classification method and system, including dividing a text sentence into words, mapping each word into a word vector; extracting keywords in the text sentence, constructing a word vector attention matrix, a position attention matrix and a part-of-speech attention matrix according to the word vector of the keyword, the position of the keyword in the text sentence and the sentiment part-of-speech type to which the keyword belongs, and fusing the three to construct a first feature; using a BiGRU network to obtain a second feature according to the contextual semantic information of the keyword; and using a multi-attention convolutional neural network model trained with the first feature and the second feature as a training set to classify the sentiment type of the text sentence to be tested. The CNN model combined with the multi-attention mechanism obtains the first dimension feature of keyword sentiment classification, obtains the second dimension feature of initial sentence sentiment classification through BiGRU, and fuses the two dimensional features to improve the perception ability of the deep semantics of the text and thus improve the accuracy of text sentiment classification.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing and deep learning technology, and in particular to a text sentiment classification method and system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the continuous development of social networks, the role of Internet users has changed from information receivers to information creators. Internet users are accustomed to expressing their opinions through the Internet and publishing content with emotional attitudes on the Internet platform, forming an expression method mainly based on short texts. The amount of text information data is large, and the arrangement of text content is scattered and messy, which is difficult to distinguish and organize manually. Therefore, how to use natural language processing related technologies to analyze the emotional polarity of short texts on social networks has become one of the hot topics today.

[0004] At present, text sentiment classification methods include feature extraction and classification based on machine learning; deep learning is an important branch of machine learning. Deep learning is an algorithm that uses multiple nonlinear transformation structures to perform high-level abstraction on data. As deep learning has achieved great success in the fields of image and speech recognition, more and more researchers have begun to use deep learning technology to solve problems related to text sentiment classification. Kim proposed an English classification model, which uses preprocessed word vectors as input and uses convolutional neural networks to achieve sentence-level classification tasks; YinW et al. used convolutional neural networks (CNN) to solve the polarity judgment problem of Twitter; although CNN has made great breakthroughs in the field of text classification, it pays more attention to local features and ignores the contextual meaning of words, which affects the accuracy of classification.

[0005] In the study of improving CNN models, Zhao W et al. proposed capsule networks and dynamic routing based on convolutional neural networks, achieving better classification results than traditional CNNs; Mikolov T et al. successively proposed the RNN model and applied it to text classification tasks. Since the output value of the current node of the RNN is determined by the current input and the output of the previous node, the word order structure of the text is taken into account, but the RNN is prone to problems such as gradient diffusion.

[0006] Although LSTM (Long Short-Term Memory) can solve the gradient diffusion problem in RNN, it has a large computational overhead due to its complex structure and stores a large number of redundant intermediate variables, which requires a lot of training time and memory space. In addition, it relies too much on historical information and cannot use future information with practical value.

[0007] BiGRU is a variant of BiLSTM and GRU. In terms of model logic structure, BiGRU simplifies and optimizes the three-gate structure of BiLSTM (forget, input, output) into two gates, namely (update and reset), so it has fewer parameters and accelerates the convergence of the model. At the same time, BiGRU combines the characteristics of short text content and high correlation with context, fully considers the meaning of words in the context, and overcomes the problem that the original GRU cannot consider the semantic information after the word.

[0008] The traditional LSTM model can effectively solve the long-range dependency of text data by introducing a recursive network structure, extract sequence semantic features from text data, and effectively avoid the problem of RNN gradient explosion. The LSTM variant BiGRU model can effectively extract the semantic information before and after the text, has a stronger perception of the overall semantics of the sentence, and the model is more streamlined than LSTM; although the BiGRU model can effectively learn the text sequence semantic features that are closely related to emotional tendency, it cannot further learn the local semantic features of the text. Summary of the invention

[0009] In order to solve the above problems, the present invention proposes a text sentiment classification method and system, which combines the CNN model with a multi-attention mechanism to obtain the first dimensional features of keyword sentiment classification, and obtains the second dimensional features of initial sentence sentiment classification through BiGRU. The two dimensional features are integrated to improve the perception ability of the deep semantics of the text and thus improve the accuracy of text sentiment classification.

[0010] In order to achieve the above object, the present invention adopts the following technical solution:

[0011] In a first aspect, the present invention provides a text sentiment classification method, comprising:

[0012] Divide the text sentence into words and map each word into a word vector;

[0013] Extract keywords from the text sentence, and construct the word vector attention matrix, position attention matrix, and part-of-speech attention matrix according to the word vector of the keyword, the position of the keyword in the text sentence, and the sentiment part-of-speech type to which the keyword belongs, and fuse the three to construct the first feature;

[0014] The BiGRU network is used to obtain the second feature based on the contextual semantic information of the keyword;

[0015] A multi-attention convolutional neural network model trained with the first feature and the second feature as the training set is used to classify the sentiment type of the test text sentence.

[0016] In a second aspect, the present invention provides a text sentiment classification system, comprising:

[0017] The segmentation module is used to divide the text sentence into words and map each word into a word vector;

[0018] The first feature extraction module is used to extract keywords from the text sentence, and construct a word vector attention matrix, a position attention matrix, and a part-of-speech attention matrix according to the word vector of the keyword, the position of the keyword in the text sentence, and the sentiment part-of-speech type to which the keyword belongs, and fuse the three to construct the first feature;

[0019] A second feature extraction module, used to obtain a second feature according to contextual semantic information of the keyword using a BiGRU network;

[0020] The classification module is used to classify the sentiment type of the text sentence to be tested by using a multi-attention convolutional neural network model trained with the first feature and the second feature as a training set.

[0021] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in the first aspect is performed.

[0022] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method described in the first aspect is performed.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The present invention proposes a MATT-CNN+BiGRU fusion model, which can not only utilize the advantages of multi-attention CNN in feature extraction and local feature extraction of target keywords, but also combines the advantages of the BiGRU model with a simple structure and taking into account the global features of the text to fully consider the contextual semantic information of the words; wherein the multi-attention mechanism CNN is combined with a specific target sentiment classification method to extract the sentiment polarity of the target keyword in the sentence, and BiGRU analyzes the sentence-level sentiment polarity, and the features of the two are fused to construct a fused global feature vector, thereby improving the fusion model's perception of the deep semantics of the text and thereby improving the accuracy of text sentiment classification.

[0025] The present invention integrates three attention mechanisms, namely word vector attention mechanism, part-of-speech attention mechanism and position attention mechanism, into a fused multi-attention mechanism and inputs it into the fusion model CNN module to improve the functional diversity of the attention mechanism; considering the influence of target keywords, the emotional feature representation of the text is enriched to make the emotional tendency judgment of the text more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0027] Figure 1 A flow chart of a text sentiment classification method provided in Example 1 of the present invention;

[0028] Figure 2 A convolutional neural network module diagram combining a multi-attention mechanism provided in Example 1 of the present invention;

[0029] Figure 3 The attention mechanism operation diagram provided in Example 1 of the present invention;

[0030] Figure 4 This is an example diagram of position calculation provided by Embodiment 1 of the present invention;

[0031] Figure 5 An example diagram of part-of-speech tagging of special words provided in Example 1 of the present invention;

[0032] Figure 6 A structural diagram of the BiGRU and multi-attention CNN (MATT-CNN) fusion model provided in Example 1 of the present invention;

[0033] Figure 7 A graph showing the change in loss function of MATT-CNN+BiGRU, traditional single CNN, and BiLSTM provided in Example 1 of the present invention;

[0034] Figure 8 A graph showing the effect of different learning rates on the performance of MATT-CNN+BiGRU under the MRD dataset provided in Example 1 of the present invention;

[0035] Fig. 9 A diagram showing the comparison of training time for different network models to complete one iteration on a Laptop dataset provided in Example 1 of the present invention;

[0036] Fig.10 A relationship diagram between the classification accuracy and the cumulative contribution rate of the characteristic values ​​provided in Example 1 of the present invention;

[0037] Fig.11A comparison chart of the classification accuracy of part-of-speech vectors of different dimensions provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0039] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0040] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0042] Example 1

[0043] The target keyword plays an important role in the field of sentiment analysis of short texts at the sentence level, and even determines the sentiment judgment of the entire sentence to a certain extent. Therefore, this embodiment takes into account the influence of the target keyword, enriches the sentiment feature representation of the text, and makes the judgment of the sentiment tendency of the text more accurate; currently in the field of specific target sentiment classification, the convolutional neural network model based on the attention mechanism has been proven to have achieved good results in semantic fine-grained target keyword extraction; based on this, based on the characteristics of the above two models, this embodiment proposes a MATT-CNN+BiGRU text sentiment classification model, and establishes two dimensions of sentiment classification: that is, the CNN (MATT-CNN) model combined with the multi-attention mechanism obtains the first dimension of keyword sentiment classification, and the second dimension of initial sentence sentiment classification is obtained through BiGRU, and the sentiment classification results of the two dimensions are spliced ​​to construct a fusion model, so as to improve the perception ability of the fusion model to the deep semantics of the text and thus improve the accuracy of text sentiment classification.

[0044] like Figure 1 As shown, this embodiment provides a text sentiment classification method, including:

[0045] S1: Divide the text sentence into words, map each word into a word vector, and construct a word vector matrix;

[0046] S2: Extract keywords from the text sentence, use the keyword's word vector as the initial attention matrix, and construct the word vector attention matrix with the word vector matrix;

[0047] S3: Use a bidirectional scanning algorithm to determine the position value between each word and the keyword, and construct a position attention matrix based on the position vector and word vector matrix obtained by mapping the position value;

[0048] S4: construct the part-of-speech attention matrix based on the sentiment part-of-speech categories of the keywords;

[0049] S5: Construct the first feature by word vector attention matrix, position attention matrix and part-of-speech attention matrix;

[0050] S6: Use the BiGRU network to obtain the second feature based on the contextual semantic information of the keyword;

[0051] S7: A multi-attention convolutional neural network model trained with the first feature and the second feature as the training set is used to classify the sentiment category of the test text sentence.

[0052] In the step S1: after the text sentence is preprocessed, a word sequence is constructed with words as units, and each word is mapped into a multi-dimensional continuous value word vector to obtain a word vector matrix;

[0053] For a sentence of length n, s = {w1, w2, ...t vital ,…w n}, where t vital is the keyword extracted by the keyword extraction algorithm, that is, the target keyword; the sentence is formed into a word sequence in units of words, and then each word is mapped to a multi-dimensional continuous value word vector to obtain the word vector matrix E∈R k×|v| , where k is the word vector dimension, that is, each word is mapped to a k-dimensional vector x i ∈R k , |V| is the dictionary size, that is, the number of all words contained in the data set;

[0054] For a sentence of length n, it is represented as a matrix:

[0055]

[0056] in, For the concatenation operation, the word vector set {x1,x2,…,x n} and keyword set {t i ,t j} to determine the sentiment polarity of each target in the target set.

[0057] The category is represented by a matrix formed by embedding the category feature words, and the relevance between text keywords and categories is evaluated by combining tf-idf and cross entropy. Given a document set about a category, p = [p 1 ,p 2 ,...p l ], in each category, the keywords with the highest correlation coefficient are selected as the category feature words, that is:

[0058]

[0059] S=[s1,s2,…,s L ]

[0060] Among them, S is the category matrix, is the concatenation operation, and s is the category keyword vector.

[0061] This embodiment adopts the following algorithm 1. First, NLTK is used to segment each text, and word2vec is used to generate word vectors after segmentation, that is, line 3; lines 4-5 use the tf-idf method to obtain the high-frequency words of each category of articles, and the high-frequency words of this category are compared with the high-frequency words of other categories through cross entropy to determine whether the high-frequency words of this category appear frequently in other categories. Line 6 obtains k keywords about this category, and finally splices them into a keyword text feature vector;

[0062]

[0063]

[0064] The attention mechanism was first applied in the field of image processing. In 2014, Mnih et al. proposed to use the attention mechanism in image classification tasks and achieved good experimental results. In 2015, Yin et al. proposed a convolutional neural network based on a multi-layer attention mechanism and applied the network to sentence modeling, which enabled it to better capture local text features and verified the effectiveness of the combination of the attention mechanism and the convolutional neural network. In this embodiment, Figure 2As shown in the figure, the multi-attention convolutional neural network model MATT-CNN is used. In aspect-based sentiment analysis (ABSA), compared with sentence-level text sentiment classification, it is a fine-grained sentiment classification task. As an important subtask of sentiment analysis, it is a deeper level of sentiment analysis, and its goal is to identify the sentiment polarity (positive, neutral, negative) in different aspect contexts. The convolutional neural network model based on the attention mechanism has been proven to achieve good results in semantically fine-grained target keyword extraction.

[0065] In step S2, a word vector attention mechanism is constructed, the word vector of the target keyword is extracted as its attention matrix, and the attention matrix and the word vector matrix are operated;

[0066] In order to better learn the feature information of different targets and identify the sentiment polarity of different targets, this embodiment uses the word vector attention mechanism and the position distance attention mechanism to focus on learning different information that needs attention in different ways.

[0067] For a sentence containing t keywords, it is divided into t clauses, and the positions are marked with special symbols "". Since the data set used in the experimental part of this embodiment is the semEval2014 data set, the target words have been marked, which can reduce the workload of keyword extraction.

[0068] The purpose of the word vector attention mechanism is to allow the model to focus on different aspects during the training process and understand which part of the information is important, so that the model can pay close attention to this information. For specific target analysis, content-level information is crucial.

[0069] For a sentence s={w1,w2,…t i ,…,w n}, extract the keyword t i The word vector of s is used as the attention matrix. The attention matrix and the word vector matrix of s are operated as shown in the following formula to obtain the attention feature matrix A c The operation process is as follows Figure 3 As shown, A c is a diagonal matrix:

[0070]

[0071] Among them, α is an adjustable parameter, similar to the learning rate η in the neural network, which is used to control the influence of different word vectors on the target keyword; α can be manually set at the beginning, or it can be calculated by the position attention mechanism. α can be used to indicate the importance of each word.

[0072] Using the calculated attention feature matrix A c By operating with the original word vector, we can get the input matrix of the convolutional neural network:

[0073]

[0074]

[0075] In step S3, constructing the position attention matrix includes:

[0076] In this embodiment, take the sentence "The《atmosphere》is magnificent,but the《performance》of the actors are awful." as an example. According to previous understanding, sentiment words close to the target keyword often have a greater impact on it. For example, for the target keyword atmosphere, the sentiment word magnificent is undoubtedly closer to it and is also a correct expression of it. However, for the target word performance, the result is not entirely true, because its distance from the sentiment word awful that describes itself is greater than magnificent. Figure 4 As shown in the figure, in the specific target sentiment classification, the position between the word and the target keyword often hides key information, so its true sentiment content cannot be obtained;

[0077] To address this problem, this embodiment proposes a bidirectional scanning algorithm to determine the position value between the word and the target, effectively identifying the degree of different words in the sentence, so that the convolutional neural network can make full use of the position information of the extracted keywords; Algorithm 2 is as follows:

[0078]

[0079] This embodiment uses the matrix ψ to store the values ​​of all positions of the data set, and then calculates the value of β:

[0080]

[0081] Map all the position values ​​stored in the matrix ψ into a multidimensional vector, namely ψ i ∈R k , the input matrix calculated is:

[0082]

[0083] It can be understood that the position attention matrix can be added to the word vector attention matrix in the form of parameters to assist its operation; it can also be input into the network as an independent attention mechanism, combining the position attention matrix with other attention matrices to fully represent the importance of each word in the sentence.

[0084] In step S4, a part-of-speech attention matrix is ​​constructed to supplement the word vector attention mechanism, associate the parts of speech of words in the sentence, and analyze the parts of speech of the text content, so as to learn more hidden information;

[0085] In this embodiment, in some data sets with low coverage of sentiment words and abnormal word segmentation, the accuracy of sentiment classification based solely on text content information to classify text sentiment will be greatly reduced; the part-of-speech attention mechanism described in this embodiment can allow the fusion model to deepen the correlation between target keywords and sentiment words by re-labeling the part-of-speech of special words.

[0086] Take the sentence "The atmosphere is magnificent, but the performance of the actors are awful" as an example. Figure 5 As shown in the figure, only the key target word "atmosphere" is analyzed, where DA represents the definite article, LV represents the linking verb, Tar represents the target keyword, CC represents the transitional preposition, Positive represents the sentiment word with positive sentiment polarity, and Negative represents the negative sentiment word; like the word vector, each sentiment word is mapped into a multi-dimensional continuous value vector, called the part-of-speech vector, i.e., Target i ∈R l , where l is the dimension of the part-of-speech vector; assuming a sentence of length m, the part-of-speech vector is represented as a vector matrix:

[0087]

[0088] Among them, Tar is the part-of-speech vector of the target keyword.

[0089] The part-of-speech vector of the extracted target keyword is used as the part-of-speech attention feature matrix, namely A t =Tar, calculate the input matrix of the network:

[0090]

[0091] Among them, β is the weight coefficient value. By adjusting the value of β, the emotional characteristics of the sentence can be fully utilized.

[0092] In the step S5, the word vector attention matrix, the position attention matrix and the part-of-speech attention matrix are fused, and different operations are used to form a fused representation of the input text as the input of the convolutional neural network part of the fusion model. In this embodiment, sliding windows of three window sizes are combined with a multi-attention mechanism to perform convolution operations;

[0093] In specific implementations, the CNN network can perform convolution calculations on multiple words in a text simultaneously through convolution kernels, preserving the associations between words;

[0094] For a sliding window of length h, the convolution operation on the input matrix is:

[0095] C i =f(w·x i:i+h-1 +b)

[0096] Where W∈R h×k is the convolution kernel weight, b∈R is the bias value, f is the activation function, x i:i+h-1 is the local feature matrix of the text of a convolution window;

[0097] For a sentence of length n, the convolution kernel operation can obtain the following feature map:

[0098] c=[c1,c2…,c n-h+1 ]

[0099] That is, c∈R n-h+1 .

[0100] The multi-attention CNN module (MATT-CNN) performs downsampling operation through the k-max pooling layer, extracts the most important information of different attention mechanisms from the pooling layer, and uses the Softmax classifier to output the first dimension result of sentiment classification considering the influence of keywords.

[0101] This example uses the k-max pooling method for downsampling:

[0102]

[0103] Among them, h represents the height of the convolution kernel, that is, the sliding window, and s represents the length of the short text sentence (controlled within 30 characters). Compared with the maximum pooling strategy, the k-max method can dynamically extract multiple important semantic combination features based on the characteristics of the multi-sliding window convolution layer, and retain the relative order relationship between the features.

[0104] In step S6, the BiGRU network is used to output the second dimension feature of sentence-level sentiment classification according to the contextual semantic information of the keyword;

[0105] In the specific implementation, the first layer of the BiGRU module is the word embedding layer, and the sentence matrix of the embedding layer is used as input; the word vector dimension set in this embodiment is consistent with that of the MATTT-CNN module, both of which are 100 dimensions, and other parameters remain unchanged.

[0106] BiGRU's bidirectional scanning operation is used to input the fusion model from two directions respectively. The information in two directions, namely historical information and future information, is saved through the hidden layer. Finally, the two hidden layer output parts are spliced ​​to obtain the final BiGRU output. The code under the Tensorflow framework is as follows:

[0107] Output_bgru=rnn.static_bidirectional_rnn(fw,bw,inputs)

[0108] In step S7, the global feature representation of the fusion model is reduced in dimension using the PCA method, and the text classification result of the convolutional neural network module of the fusion model is output through the softmax function, such as Figure 6 shown.

[0109] In the specific implementation, the feature vector output by the downsampling layer is used as the input of the fully connected layer, and the proposed model samples the softmax function to output the classification result:

[0110] y=softmax(W f X P +B f )

[0111] Among them, X p is the output of the downsampling layer, is the weight matrix of the fully connected layer, B f ∈R C is the bias value of the full connection; the loss function is optimized by cross entropy, and the cross entropy cost function is:

[0112]

[0113] Where D is the size of the training data set, C is the number of categories, and y is the predicted category. is the actual category. λ||θ|| 2 is the regularization term.

[0114] The fused features are saved in the output as the input of the first fully connected layer. The dropout mechanism is introduced in the two FC layers to prevent overfitting. Each iteration will abandon some trained parameters so that the weight update no longer depends on some inherent features. Before the feature fusion vector is output through the softmax classifier, the PCA method introduced earlier is used to reduce the dimension, retain the main useful information, and thus improve the convergence speed of the model. The probability of classifying x as in the softmax regression in the model of this embodiment is:

[0115]

[0116] In order to verify the effectiveness of the model proposed in this embodiment, two datasets, MRD (Movie Review Data) created by Cornell University film review data and SemEval2016, were used. Various experiments proved that compared with the currently widely used attention sentiment classification model, the MATT-CNN+BiGRU fusion model proposed in this embodiment can improve the accuracy while reducing the training time of the model;

[0117] In the experiment, various window convolution kernels are used to perform convolution operations on the input matrix. The convolution kernel function is rectified linear units. The Adadelta update rule proposed by Zeiler is used during training. Other hyperparameters are shown in Table 1:

[0118] Table 1 Experimental parameters

[0119]

[0120]

[0121] The accuracy index (Accuracy) is used to measure the performance of the sentiment classification algorithm, and its calculation method is:

[0122]

[0123] Among them, TP represents the number of samples that are actually positive and predicted to be positive; FN represents the number of samples that are actually positive and predicted to be negative; TN represents the number of samples that are actually negative and predicted to be negative; FP represents the number of samples that are actually negative and predicted to be positive.

[0124] Figure 7 The loss function comparison between the MATT-CNN+BiGRU model and the traditional single CNN and BiLSTM is shown; Figure 8The figure shows the effect of different learning rates on the performance of MATT-CNN+BiGRU under the MRD dataset. After comparison, the traditional CNN model has a lower loss value than the iterative stable value of the BiLSTM and MATT-CNN+BiGRU loss functions, but the MATT-CNN+BiGRU model decreases faster than the BiLSTM, and the final loss values ​​of the three models all drop to a very low value, with good convergence effect.

[0125] In the gradient descent algorithm, if the initial value of the learning rate is set too small, it will lead to an increase in the number of iterations, or fall into a local optimal solution to form an infinite loop; if the learning rate is set too large, it will cause the cost function to oscillate and become unstable, unable to approach the actual minimum value, and slow down the convergence of the algorithm.

[0126] In order to verify the positive impact of PCA on the training of feature fusion vectors after dimensionality reduction, different network models were analyzed to complete all experiments under the same CPU, GPU and framework, while other hyperparameters such as word vector construction methods were also kept consistent.

[0127] Fig. 9 The comparison of the training time of different network models for one iteration on the Laptop dataset shows that the training time cost of the LSTM network is very high. This is mainly because the LSTM network is trained on sequential data. The training time for one iteration on Restaurant and Laptop is 437s and 500s respectively.

[0128] In addition, CNN without any attention mechanism is the fastest, taking only 4s and 5s, which is consistent with the conventional understanding of CNN. The MATT-CNN+BiGRU model proposed in this embodiment changes the original cumbersome LSTM model, and has obvious improvements over LSTM on both data sets. After adding PCA dimensionality reduction, the training time of the MATT-CNN+BiGRU model on the two data sets was reduced to 48s and 53s, respectively. This verifies the positive impact of PCA on the training overhead of the MATT-CNN+BiGRU model proposed in this embodiment.

[0129] In order to study the impact of the cumulative contribution rate of the principal component eigenvalue on the classification performance of the MATT-CNN+BiGRU model, the relationship between different cumulative contribution rates and the classification accuracy of the MATT-CNN+BiGRU model was analyzed under the MRD dataset. The relationship between the classification accuracy and the cumulative contribution rate of the eigenvalue is shown in the figure. Fig.10As shown in the figure, let the cumulative contribution rate of the eigenvalue be α. When α increases from 100% to 95%, the text features extracted by the MATT-CNN module are processed by PCA dimensionality reduction, the redundancy in the text features is gradually eliminated, and the classification accuracy increases; when α = 95%, the redundant information in the text features is fully eliminated, and the classification accuracy is also the highest; when α gradually decreases from 95%, some of the used texts are also eliminated, resulting in a decrease in classification accuracy. It can be concluded that the dimension of the principal component is crucial to the classification accuracy.

[0130] This embodiment adds a part-of-speech attention mechanism. In order to verify the effectiveness of the part-of-speech attention mechanism, 2500 and 2000 data are extracted from the MRD dataset and the Restaurant dataset respectively for verification experiments; among them, the part-of-speech vector dimension of 0 indicates that the part-of-speech attention mechanism is not applicable. The results are as follows Fig.11 After adding the part-of-speech attention mechanism, the classification effect of the MATT-CNN+BiGRU model has been significantly improved in both datasets; calculated by peak value, MATT-CNN+BiGRU has improved by 8.9% on the MRD dataset and 9.9% on the Restaurant dataset; it proves that adding the part-of-speech attention mechanism can make the model learn the sentiment information of the text more fully and achieve better sentiment classification effect.

[0131] When the word vector is greater than 100 dimensions, the model fluctuates in both data sets; after analysis, the reason is that the input matrix of the MATT-CNN+BiGRU model is the input associated with the attention mechanism. Word vector attention is the main feature of model training. When the dimension of the part-of-speech vector exceeds a certain threshold, it will affect its parameter adjustment and reduce the learning effect of content-level feature information; in addition, as the vector dimension increases, the training cost of the model will also increase accordingly. Therefore, in the above experiment, this embodiment uses the empirical parameter value 100 as the dimension of the part-of-speech attention vector.

[0132] Example 2

[0133] This embodiment provides a text sentiment classification system, including:

[0134] The segmentation module is used to divide the text sentence into words and map each word into a word vector;

[0135] The first feature extraction module is used to extract keywords from the text sentence, and construct a word vector attention matrix, a position attention matrix, and a part-of-speech attention matrix according to the word vector of the keyword, the position of the keyword in the text sentence, and the sentiment part-of-speech type to which the keyword belongs, and fuse the three to construct the first feature;

[0136] A second feature extraction module, used to obtain a second feature according to contextual semantic information of the keyword using a BiGRU network;

[0137] The classification module is used to classify the sentiment type of the text sentence to be tested by using a multi-attention convolutional neural network model trained with the first feature and the second feature as a training set.

[0138] Wherein, the first feature extraction module includes:

[0139] The first matrix module extracts keywords from the text sentence, uses the keyword's word vector as the initial attention matrix, and constructs a word vector attention matrix with the word vector matrix;

[0140] The second matrix module uses a bidirectional scanning algorithm to determine the position value between each word and the keyword, and constructs a position attention matrix based on the position vector and word vector matrix obtained by mapping the position value;

[0141] The third matrix module constructs a part-of-speech attention matrix based on the sentiment part-of-speech of the keywords obtained from the sentiment part-of-speech category;

[0142] The fusion module constructs the first feature by the word vector attention matrix, position attention matrix and part-of-speech attention matrix.

[0143] It should be noted that the above modules correspond to steps S1 to S7 in Example 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above Example 1. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer executable instructions.

[0144] In further embodiments, there is also provided:

[0145] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in Embodiment 1 is performed. For the sake of brevity, it will not be described in detail here.

[0146] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0147] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0148] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the method described in Example 1 is completed.

[0149] The method in Example 1 can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it is not described in detail here.

[0150] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in conjunction with this embodiment can be implemented in electronic hardware or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0151] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0152] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A text sentiment classification method, characterized in that: include: Divide the text sentence into words and map each word into a word vector; Extract keywords from text sentences, and construct a word vector attention matrix, a position attention matrix, and a part-of-speech attention matrix according to the word vector of the keyword, the position of the keyword in the text sentence, and the emotional part-of-speech type to which the keyword belongs, and fuse the three to construct the first feature; wherein, the word vector construction matrix construction process is: construct a word vector matrix according to each word vector, take the word vector of the keyword as the initial attention matrix, and construct a word vector attention matrix with the word vector matrix; the position attention matrix construction process is: use a bidirectional scanning algorithm to determine the position value between each word and the keyword, obtain the position vector by mapping the position value, and construct a position attention matrix according to the word vector matrix and the position vector constructed by each word vector; the part-of-speech attention matrix construction process is: construct the emotional part of speech of the keyword obtained according to the emotional part-of-speech category A part-of-speech attention matrix, wherein the emotional part-of-speech categories include but are not limited to transitional prepositions, positive emotional words, and negative emotional words; the word vector attention matrix, the position attention matrix, and the part-of-speech attention matrix are fused, the position attention matrix is ​​added to the word vector attention matrix in the form of a parameter to assist its operation or is input into the network as an independent attention mechanism, the position attention matrix and the other attention matrices are combined to form a fused representation of the input text, which is used as the input of the convolutional neural network model part in the multi-attention convolutional neural network model, the multi-attention convolutional neural network model simultaneously performs convolution calculations on multiple words of the text sentence through a convolution kernel, performs downsampling operations through a k-max pooling layer, extracts features of the three attention matrices from the pooling layer, and uses a Softmax classifier to output the first dimension features that consider the influence of keywords; The BiGRU network is used to obtain the second dimension features based on the contextual semantic information of the keywords; The sentiment classification results of the two dimensional features are concatenated to construct the fusion model MATT-CNN+BiGRU, and the multi-attention convolutional neural network model trained with the first and second features as training sets is used to classify the sentiment type of the test text sentence.

2. A text sentiment classification method as claimed in claim 1, characterized in that: The keyword extraction algorithm is used to extract keywords from text sentences, including: For each word vector, a combination of Tf-idf and cross entropy is used to determine the k words with the highest frequency in each category of feature words, which are the keywords.

3. A text sentiment classification method as claimed in claim 1, characterized in that: After the first feature and the second feature are fused, the PCA dimensionality reduction technique is used to reduce the dimensionality of the fused feature global vector, and the softmax classifier is used to output the final sentiment classification result.

4. A system according to any one of claims 1 to 3, characterized in that: include: The segmentation module is used to divide the text sentence into words and map each word into a word vector; The first feature extraction module is used to extract keywords from text sentences, and construct a word vector attention matrix, a position attention matrix and a part-of-speech attention matrix according to the word vector of the keyword, the position of the keyword in the text sentence and the emotional part-of-speech type to which the keyword belongs, and fuse the three to construct the first feature, wherein the word vector construction matrix construction process is: construct a word vector matrix according to each word vector, take the word vector of the keyword as the initial attention matrix, and construct a word vector attention matrix with the word vector matrix; the position attention matrix construction process is: use a bidirectional scanning algorithm to determine the position value between each word and the keyword, obtain the position vector by mapping the position value, and construct a position attention matrix according to the word vector matrix and the position vector constructed by each word vector; the part-of-speech attention matrix construction process is: according to the keyword obtained according to the emotional part-of-speech category The sentiment part-of-speech attention matrix is ​​constructed based on the sentiment part-of-speech categories, including but not limited to transitional prepositions, positive sentiment words, and negative sentiment words; the word vector attention matrix, the position attention matrix, and the part-of-speech attention matrix are fused, and the position attention matrix is ​​added to the word vector attention matrix in the form of a parameter to assist its operation or input into the network as an independent attention mechanism, and the position attention matrix and other attention matrices are combined to form a fused representation of the input text as the input of the convolutional neural network model part in the multi-attention convolutional neural network model, and the multi-attention convolutional neural network model simultaneously performs convolution calculations on multiple words of the text sentence through a convolution kernel, performs downsampling operations through a k-max pooling layer, extracts features of the three attention matrices from the pooling layer, and uses a Softmax classifier to output the first feature that considers the influence of keywords; A second feature extraction module, used to obtain a second feature according to contextual semantic information of the keyword using a BiGRU network; The classification module is used to concatenate the sentiment classification results of the two dimensional features to construct a fusion model MATT-CNN+BiGRU, and adopts a multi-attention convolutional neural network model trained with the first feature and the second feature as the training set to classify the sentiment type of the text sentence to be tested.

5. An electronic device, characterized in that: The method comprises a memory and a processor and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 3 is completed.

6. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the method described in any one of claims 1 to 3.

Images