A text sentiment analysis method, system, device and storage medium

Abstract

The application provides a text sentiment analysis method, system, device and storage medium, and belongs to the field of natural language processing, which comprises the following steps: obtaining a text to be analyzed; constructing a sentiment analysis model, the feature extraction layer of the sentiment analysis model being obtained through self-attention mechanism, simple recurrent unit and capsule network fusion; inputting the text to be analyzed into the sentiment analysis model, the input layer of the sentiment analysis model converting the text to be analyzed into a text word vector; the self-attention mechanism of the feature extraction layer performing global semantic feature extraction on the text word vector, the simple recurrent unit performing context semantic feature extraction on the text word vector, and the capsule network performing local semantic feature extraction on the text word vector; the feature fusion layer performing feature fusion on the extracted global semantic feature, context semantic feature and local semantic feature to obtain sentiment information features at different levels; and the output layer outputting the sentiment information features at different levels to obtain the analyzed text.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of natural language processing, and particularly relates to a text sentiment analysis method, system, device and storage medium. BACKGROUND

[0002] With the rapid development of the Internet and social media, Chinese text sentiment analysis has become increasingly important in the field of natural language processing. In the digital age, people communicate and express emotions in the virtual world through text, and Chinese text sentiment analysis has become a key tool for interpreting these emotions.

[0003] Traditional sentiment analysis methods, such as dictionary-based methods and rule-based methods, can only identify the overall sentiment tendency of the text, i.e., determine whether the text is positive, negative, or neutral. However, this method cannot meet the needs of detailed and in-depth characterization of emotions in the text. In addition, the expression of emotions in Chinese text is rich and complex. As a vast and profound language, Chinese has diverse and delicate ways of expressing emotions. The same word can express completely different emotions in different contexts, and the same emotion can be expressed through multiple words and sentence patterns, making the task of Chinese text sentiment analysis more challenging.

[0004] In existing solutions, local feature extraction is mostly performed by convolutional networks, and context semantic feature extraction is performed by long short-term memory networks or gated recurrent units. However, convolutional networks capture local features through filters, and these features are static and lack hierarchy, which limits the understanding and expression of hierarchy and compositionality in text. Long short-term memory networks or gated recurrent units introduce a gating mechanism to control long-term dependencies of information, which makes the network structure complex, increases the number of parameters, and increases the computational burden and training difficulty of the model. SUMMARY

[0005] In order to overcome the deficiencies of the prior art, the present application provides a text sentiment analysis method, comprising the following steps:

[0006] Obtaining the text to be analyzed;

[0007] Constructing a sentiment analysis model, the sentiment analysis model comprising an input layer, a feature extraction layer, a feature fusion layer, and an output layer, wherein the feature extraction layer is obtained by self-attention mechanism, simple recurrent unit, and capsule network fusion;

[0008] The text to be analyzed is input into the sentiment analysis model, and an input layer of the sentiment analysis model converts the text to be analyzed into a text word vector; a self-attention mechanism of a feature extraction layer performs global semantic feature extraction on the text word vector, a simple recurrent unit performs context semantic feature extraction on the text word vector, and a capsule network performs local semantic feature extraction on the text word vector; a feature fusion layer performs feature fusion on the extracted global semantic features, context semantic features and local semantic features to obtain sentiment information features at different levels; and an output layer outputs the sentiment information features at different levels to obtain the analyzed text.

[0009] Preferably, the text word vector is obtained by inputting the text to be analyzed into a pre-trained model BERT-wwm in the input layer.

[0010] Preferably, before the text to be analyzed is input into the sentiment analysis model, the sentiment analysis model is trained, including the following steps:

[0011] A text data set is obtained.

[0012] The text in the text data set is input into the sentiment analysis model, and a focal loss function is used to train the sentiment analysis model.

[0013] Preferably, the context semantic feature extraction specifically includes: using a forget gate and an update gate of a simple recurrent unit to control the forgetting degree and the updating degree of input information respectively, determining the updating degree of the input information, adding the proportions of updated information and non-updated information to obtain the output of the simple recurrent unit, and combining the output of the simple recurrent unit with the self-attention mechanism to obtain a final context semantic vector.

[0014] Preferably, the capsule network performs local semantic feature extraction on the text word vector, specifically: the input text word vector is input into a vector space with the same number of neurons as the capsule network, and weight distribution is performed to obtain local semantic features.

[0015] The application also provides a text sentiment analysis system, including:

[0016] A text acquisition module is configured to acquire text to be analyzed.

[0017] A model construction module is configured to construct a sentiment analysis model, and the sentiment analysis model includes an input layer, a feature extraction layer, a feature fusion layer and an output layer, wherein the feature extraction layer is obtained by fusing a self-attention mechanism, a simple recurrent unit and a capsule network.

[0018] The feature extraction module is used for inputting a text to be analyzed into a sentiment analysis model, an input layer of the sentiment analysis model converts the text to be analyzed into a text word vector, a self-attention mechanism of a feature extraction layer performs global semantic feature extraction on the text word vector, a simple recurrent unit performs context semantic feature extraction on the text word vector, and a capsule network performs local semantic feature extraction on the text word vector, a feature fusion layer performs feature fusion on the extracted global semantic feature, context semantic feature and local semantic feature to obtain sentiment information features at different levels, and an output layer outputs the sentiment information features at different levels to obtain the analyzed text.

[0019] The application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the text sentiment analysis method.

[0020] The application further provides a computer readable storage medium storing a computer program, wherein the computer program is adapted to be loaded by a processor to execute the text sentiment analysis method.

[0021] The text sentiment analysis method, system, device and storage medium provided by the application have the following beneficial effects:

[0022] The feature extraction layer of the sentiment analysis model constructed by the application can perform multi-dimensional and deep feature mining on the input text by fusing the self-attention mechanism, simple recurrent unit and capsule network, better handle the structure and semantic relationship in the text data, make the obtained text have more levels, facilitate the understanding of the text, and in addition, the simple structure and direct feedback mechanism of the simple recurrent unit can reduce the model parameters, improve the interpretability of the model, thereby speeding up the training process and reducing the demand for computing resources. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the application and the design scheme thereof, the drawings required by the embodiments will be briefly introduced as follows. The drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0024] Figure 1 The text sentiment analysis method of the embodiment of the application is shown in the flowchart.

[0025] Figure 2 The text sentiment analysis method of the embodiment of the application is shown in the flowchart. DETAILED DESCRIPTION

[0026] In order for those skilled in the art to better understand the technical solutions of the present application and to implement them, the present application will be described in detail below in conjunction with the drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and cannot be used to limit the protection scope of the present application.

[0027] Embodiments

[0028] The present application provides a text sentiment analysis method, specifically as shown in Figure 1 , comprising the following steps:

[0029] Step 1: Obtain the text to be analyzed.

[0030] Step 2: Build a sentiment analysis model, which includes an input layer, a feature extraction layer, a feature fusion layer, and an output layer. The feature extraction layer is obtained through a self-attention mechanism, a simple recurrent unit, and a capsule network fusion.

[0031] Step 3: Input the text to be analyzed into the sentiment analysis model. The input layer of the sentiment analysis model converts the text to be analyzed into a text word vector. The self-attention mechanism of the feature extraction layer extracts global semantic features from the text word vector, the simple recurrent unit extracts context semantic features from the text word vector, and the capsule network extracts local semantic features from the text word vector. The feature fusion layer fuses the extracted global semantic features, context semantic features, and local semantic features to obtain sentiment information features at different levels. The output layer outputs the sentiment information features at different levels to obtain the analyzed text.

[0032] As shown in Figure 2 , wherein the text to be analyzed is input into the pre-trained model BERT-wwm to obtain the text word vector , the obtained text word vector is input into the feature extraction layer, and the self-attention mechanism, the simple recurrent unit, and the capsule network of the feature extraction layer are used to obtain the global sentiment vector , the context sentiment vector , and the local sentiment vector , respectively, wherein the obtained context sentiment vector is combined with the self-attention mechanism to obtain the final context semantic vector , the flattening layer merges the local sentiment vectors of multiple capsule networks and flattens them into a vector, which is then fused with the global sentiment vector and the final context semantic vector in the fully connected layer, and output through the output layer.

[0033] The specific process is as follows:

[0034] (1) Train the sentiment analysis model, and the training process is as follows:

[0035] Obtain a text dataset.

[0036] Input the text in the text dataset into a sentiment analysis model, and train the sentiment analysis model using a focal loss function, wherein the focal loss function is:

[0037]

[0038] In the formula, p i represents a predicted probability value of the sentiment analysis model, gamma is an adjustment parameter of the sentiment analysis model, a i is a balance parameter of the sentiment analysis model. When the predicted probability of the sentiment analysis model for a certain sample is high, p i approaches 1, approaches 0, thereby reducing the loss weight of easy-to-classify samples and increasing the attention to difficult-to-classify samples, thereby improving the learning effect on these samples; when the sample category imbalance is serious, the attention to minority class samples can be increased by adjusting a i , thereby improving the learning effect of the model on the minority class samples.

[0039] Introducing the focal loss function to train the model can solve the problem of unbalanced distribution of sample quantities.

[0040] Input the text to be analyzed into the sentiment analysis model, and the input layer of the sentiment analysis model converts the text to be analyzed into a text word vector, specifically:

[0041] The text to be analyzed is input into BERT-wwm, and the text to be analyzed is encoded by BERT-wwm to obtain a word vector, a word vector matrix and a last layer output vector containing the semantics of the entire sentence, wherein L is the number of words in the input text, that is, the length of the text; is the word vector dimension, and each row of the word vector matrix represents a vector mapped by a single word.

[0042] Since the length of each text is not the same, in order to reduce the complexity of the sentiment analysis model and improve the efficiency and accuracy of the analysis, in this embodiment, a "long cut short supplement" method is used to process the text length, that is, if the text length exceeds the specified length limit, the text is cut off to only keep the front part of the text; if the text length is insufficient, the text length is supplemented with a number 0, so that the text length is unified to 129, that is, L = 129. ​

[0043] In this embodiment, the training batch size is set to 16, the Adam optimizer is selected for parameter optimization, and the initial learning rate is set to 1e -5 The "early stopping" mechanism is used to prevent overfitting, that is, if the accuracy does not improve after 10 rounds, the training is stopped, and the word vector dimension obtained after using BERT-wwm for word vector encoding is 768, the simple recurrent unit dimension is 256, the number of capsules in the capsule network is 6, the capsule layer hidden state dimension is 128, and the dynamic routing iteration number is 5.

[0044] (3) Use the sentiment analysis model to extract features from the text vector, including global semantic feature extraction, context semantic feature extraction, and local semantic feature extraction. Among them, the global semantic feature extraction is performed by outputting the last layer of BERT-wwm to the self-attention mechanism; the context semantic feature extraction is performed by combining the simple recurrent unit with the self-attention mechanism; in the local semantic feature channel, the lowest layer of the capsule receives a small area of the input text as the input, and uses the weight matrix to perform affine transformation on the input vector, and maps the input vector to a vector space with the same number of neurons as the capsule layer. Through multiple iterations of dynamic routing, the deep features of the input text sequence are effectively extracted.

[0045] First, in global semantic feature extraction, the hidden state corresponding to the[ CLS ]position in the output of the last layer of BERT-wwm is usually used as the representation of the entire sentence or text,[ CLS ] is a special semantic feature vector in BERT-wwm that can represent the entire sentence text, therefore, the vector generated by combining the self-attention mechanism is selected as the global semantic vector a n The calculation formula is:

[0046] ,

[0047] Among them, W Q , W K , W V are weight matrices for linear transformation, and the higher the weight, the more important the word, is a scaling factor to prevent the vector dot product from being too large, is the output vector of the last layer of the input layer,[ CLS ] means classification, and is a special semantic feature vector in BERT-wwm that can represent the entire sentence text, which can be used for classification tasks.

[0048] Secondly, in the context of semantic feature extraction, first through linear change the input data x t to the hidden state , the mathematical expression is:

[0049] ,

[0050] where, W is the weight of the text input.

[0051] f t is the "forget gate", used to control the degree of information forgetting, the calculation formula is as follows:

[0052] ,

[0053] where, sigma is the Sigmoid function, W f is the weight of the forget gate input, b f is the bias term.

[0054] "update gate" r t to control the degree of information update, the calculation formula is:

[0055] ,

[0056] where, W r is the weight of the update gate input, b r is the bias term.

[0057] the intermediate state of the transmission process c t integrates the information of the past state and the current input information, where the degree of information forgetting depends on the forget gate f t , the larger f t , the less the proportion of past information retention, the more the proportion of adding new current input information, c t The update formula is as follows:

[0058] ,

[0059] where, is the t state at the previous moment.

[0060] intermediate state c tAfter the update is completed, the update is completed tanh The activation function is activated and combined with the update gate r t Operation is performed to obtain the proportion of reserved intermediate state information; and the current input x t And the non-update degree (1- r t Operation is performed to determine the proportion of new input information reserved, and finally the proportions of the two are added to obtain the final output state h n , The specific formula is as follows:

[0061] ,

[0062] To further extract the context semantic feature information, the output of the simple cycle unit h n Combined with the self-attention mechanism to obtain the final context semantic vector m n , Realize the mining of important emotional information, m n The calculation formula is as follows:

[0063] ,

[0064] Furthermore, in the local semantic feature extraction module, the application introduces a capsule network for local feature extraction to capture the hierarchical features of the text and understand the hierarchy and combination of the text.

[0065] Specifically, first, the input is mapped to a vector space with the same number of neurons as the capsule layer:

[0066] ,

[0067] Among them, i is a low-level capsule, j is a high-level capsule, is a capsule i The reconstruction operation of the output vector of the capsule j , W ij is a weight matrix, u i is a low-level capsule output vector.

[0068] The weight distribution is performed to calculate the output vector of the low-level capsule of the capsule network S j :

[0069] ,

[0070] Among them, cij is the capsule i is the capsule j The weight between the capsules is as follows:

[0071] ,

[0072] Wherein, b ij is the capsule i is the coupling coefficient between the capsules, j b ik is the capsule i is the coupling coefficient between the capsules, k

[0073] The final output vector of the capsule network is calculated using the squeeze function g n The mathematical form is:

[0074] ,

[0075] (4) In the feature fusion layer, the feature vectors of the three channels are fused to obtain the final semantic representation vector, and the sottmax classifier is used for output.

[0076] The output vector of the last layer of BERT-wwm a n The context feature vector extracted by the simple recurrent unit m n The local semantic vector extracted by the capsule network g n Fusion, obtain different levels of emotional information features v The calculation formula is:

[0077] ,

[0078] The performance of the sentiment analysis model is verified on the SMP2020-EWECT, NLPCC2013 and NLPCC2014 public data sets, and the experimental results show that the accuracy of the method is 80.10%, 66.65% and 70.43% respectively on the three data sets, and the accuracy of the method is excellent.

[0079] The capsule network extracts local semantic features of the text word vector, specifically: the input text word vector is input into the vector space with the same number of neurons as the capsule network, and the weight distribution is performed to obtain the output vector.

[0080] ​​The application further provides a text sentiment analysis system, comprising a text acquisition module, a model construction module and a feature extraction module.

[0081] The application further provides a computer device, comprising a memory and a processor; the memory stores a computer program, and the processor is used for running the computer program in the memory to execute the text sentiment analysis method.

[0082] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is suitable for being loaded by a processor to execute the text sentiment analysis method.

[0083] The above embodiments are only the preferred specific implementation of the application, and the protection scope of the application is not limited thereto, and any simple change or equivalent replacement of the technical solution within the technical range disclosed by the application can be obtained by those skilled in the art, and all of them belong to the protection scope of the application.

Claims

1. A method of text sentiment analysis, characterized in that, The method comprises the following steps: acquiring a text to be analyzed; building a sentiment analysis model, the sentiment analysis model comprising an input layer, a feature extraction layer, a feature fusion layer and an output layer, wherein the feature extraction layer is obtained by fusing a self-attention mechanism, a simple recurrent unit and a capsule network; inputting the text to be analyzed into the sentiment analysis model, the input layer of the sentiment analysis model converting the text to be analyzed into a text word vector; the self-attention mechanism of the feature extraction layer performing global semantic feature extraction on the text word vector, the simple recurrent unit performing context semantic feature extraction on the text word vector, and the capsule network performing local semantic feature extraction on the text word vector; the feature fusion layer performing feature fusion on the extracted global semantic feature, context semantic feature and local semantic feature to obtain sentiment information features at different levels; and the output layer outputting the sentiment information features at different levels to obtain an analyzed text; the context semantic feature extraction specifically comprises: using a forget gate and an update gate of the simple recurrent unit to control the forgetting degree and the updating degree of input information respectively, determining the updating degree of the input information, adding the proportions of updated information and non-updated information to obtain the output of the simple recurrent unit, and combining the output of the simple recurrent unit with the self-attention mechanism to obtain a final context semantic vector; the method further comprises training the sentiment analysis model using a focal loss function, wherein the focal loss function is: FL(p i )=-a i (1-p i ) γ log(p i ); In the formula, p i represents the prediction probability value of the sentiment analysis model, γ is the adjustment parameter of the sentiment analysis model, α i is the balance parameter of the sentiment analysis model; In the global semantic feature extraction, the hidden state corresponding to the [CLS] position in the output of the last layer of the BERT-wwm is usually used as the representation of the entire sentence or text. The [CLS] is a special semantic feature vector representing the entire sentence text in the BERT-wwm. Therefore, the vector generated by combining the self-attention mechanism is selected as the global semantic vector a n The calculation formula is: wherein W Q , W K , W V are weight matrices for linear transformation, the higher the weight, the more important the word, is a scaling factor to prevent the vector dot product from being too large, v [CLS] is the output vector of the last layer of the input layer, [CLS] means classification, which is a special semantic feature vector representing the entire text in BERT-wwm, used for classification tasks.

2. The method of claim 1, wherein, the text word vector is obtained by performing word vector encoding on the text to be analyzed by a pre-training model BERT-wwm in the input layer.

3. The method of claim 1, wherein, Before the text to be analyzed is input into the sentiment analysis model, the sentiment analysis model is further trained, comprising the following steps: acquiring a text data set; inputting the text in the text data set into the sentiment analysis model and training the sentiment analysis model using the focal loss function.

4. The method of claim 1, wherein, The capsule network performs local semantic feature extraction on the text word vector, specifically by inputting the input text word vector into a vector space with the same number of neurons as the capsule network and performing weight distribution to obtain local semantic features.

5. A text sentiment analysis system characterized by, The method comprises the following steps: a text acquisition module for acquiring a text to be analyzed; a model building module for building a sentiment analysis model, the sentiment analysis model comprising an input layer, a feature extraction layer, a feature fusion layer and an output layer, wherein the feature extraction layer is obtained by fusing a self-attention mechanism, a simple recurrent unit and a capsule network; The feature extraction module is used for inputting the text to be analyzed into the sentiment analysis model, and an input layer of the sentiment analysis model converts the text to be analyzed into a text word vector; a self-attention mechanism of the feature extraction layer performs global semantic feature extraction on the text word vector, a simple recurrent unit performs context semantic feature extraction on the text word vector, and a capsule network performs local semantic feature extraction on the text word vector; a feature fusion layer performs feature fusion on the extracted global semantic features, context semantic features and local semantic features to obtain sentiment information features at different levels; an output layer outputs the sentiment information features at different levels to obtain the analyzed text; the context semantic feature extraction specifically comprises: using a forget gate and an update gate of the simple recurrent unit to control the forgetting degree and the updating degree of input information respectively, determining the updating degree of the input information, adding the proportion of updated information and non-updated information to obtain the output of the simple recurrent unit, and combining the output of the simple recurrent unit with the self-attention mechanism to obtain a final context semantic vector; Further comprising: training the sentiment analysis model using a focal loss function, wherein the focal loss function is: FL(p i ) = -a i log(p i ) γ log(p i ) In the formula, p i represents the prediction probability value of the sentiment analysis model, γ is the adjustment parameter of the sentiment analysis model, α i is the balance parameter of the sentiment analysis model; In the global semantic feature extraction, the hidden state corresponding to the [CLS] position in the output of the last layer of the BERT-wwm is usually used as the representation of the entire sentence or text. The [CLS] is a special semantic feature vector representing the entire sentence text in the BERT-wwm. Therefore, the vector generated by combining the self-attention mechanism is selected as the global semantic vector a n The calculation formula is: where W Q , W K , W V are weight matrices for linear transformation, the higher the weight, the more important the word, d k is a scaling factor to prevent the vector dot product from being too large, v [CLS] is the output vector of the last layer of the input layer, and [CLS] is the special semantic feature vector representing the entire text in BERT-wwm, which is used for classification tasks.

6. A computer device, comprising: The computer readable storage medium stores a computer program, and the processor is used to load the computer program in the memory to execute the text sentiment analysis method in any one of claims 1-4.

7. A computer readable storage medium characterized in that, The computer readable storage medium stores a computer program, and the processor is used to load the computer program in the memory to execute the text sentiment analysis method in any one of claims 1-4.

Images