A short text implicit sentiment classification method based on fine-grained event information enhancement

Abstract

The present invention discloses a short text implicit sentiment classification method based on fine-grained event information enhancement, the steps of which include: 1, using an event extractor to extract event types, trigger words and other event elements in a short text to form a fine-grained single event tuple; 2, using a BERT pre-training model to characterize the single event tuple and the short text respectively, and obtaining feature vectors of the single event tuple and the short text respectively; 3, fusing the feature vectors obtained in step 2 by means of tensor combination, and using fine-grained event information to enhance the representation of the short text; 4, inputting the feature vector after the fine-grained event information enhancement obtained in step 3 into a Bi-GRU model for sentiment classification, and finally outputting the result of the sentiment classification by a sigmoid network. The present invention uses fine-grained event information to enhance the representation of the short text, thereby improving the accuracy of sentiment classification.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and specifically is an implicit sentiment classification method based on enhancing short text representation with fine-grained event information. Background Art

[0002] With the development of social media, a large amount of short text information has accumulated on the Internet. Sentiment classification of these short text information can effectively explore the emotions hidden in the short texts of commentators. This can not only help decision-makers understand online public opinion, but also help businesses understand consumer preferences.

[0003] Sentiment classification tasks can be divided into explicit and implicit sentiment classification, depending on whether a text contains sentiment words. Explicit sentiment classification methods primarily detect sentiment based on the presence of sentiment-related words and have been widely explored. Compared to the widely studied explicit sentiment analysis, implicit sentiment analysis is a more difficult task due to the lack of sentiment words. Summary of the Invention

[0004] In order to address the shortcomings of the above-mentioned prior art, the present invention proposes a short text implicit sentiment classification method based on fine-grained event information enhancement, in order to classify and identify the sentiment of short texts that lack sentiment words and improve the accuracy of sentiment prediction.

[0005] In order to achieve the above-mentioned object, the present invention adopts the following technical solutions:

[0006] The present invention is characterized in that the short text implicit sentiment classification method based on fine-grained event information enhancement is carried out according to the following steps:

[0007] Step 1: Get the short texts in the comment data and build a data set. Perform data preprocessing on all the short texts in the data set to obtain the word segmentation set corresponding to each short text. The word segmentation set of any short text s = (s1, s2, ..., s i ,...,s a ), where s i represents the i-th word in the word segmentation set s, and a represents the total number of words in the word segmentation set s;

[0008] Step 2: Use the event extractor to extract single events from the word set s to extract the elements of the single event, thereby obtaining the fine-grained single event tuple t = (t1, t2, ..., t l ,...,t b ), where t l represents the lth element in the single event tuple t, b represents the total number of elements in the single event tuple t, l∈[1,b];

[0009] Step 3: Input the single event tuple t into the BERT pre-training model to obtain the hidden state of each element in the single event tuple t, where the hidden state of the lth element is recorded as H l ;

[0010] Step 4: Input the hidden state of each element in the single event tuple t into the attention mechanism network to obtain the representation vector of the single event tuple t;

[0011] Step 4.1: Use equations (1) and (2) to calculate the lth element t in the single event tuple t l The attention score α l :

[0012] u l =sigmoid(W1H l +b1) (1)

[0013]

[0014] In formula (1), W1 is the first weight matrix, b1 is the first bias vector, sigmoid represents the activation function, and u l is the lth element t in the single event tuple t l The new hidden state of

[0015] In formula (2), T represents the transposition operation, θ l Represents the randomly initialized lth element t l The attention matrix;

[0016] Step 4.2: Use formula (3) to obtain the attention feature vector h of the single event tuple t t , and as the characterization vector of the single event tuple t:

[0017]

[0018] Step 5: Input the word segmentation set s into the BERT pre-training model to obtain the hidden vector hs of the short text, and then use formula (4) to obtain the short text representation r enhanced by single event information final :

[0019]

[0020] In formula (4), W2 is the second weight matrix, b2 is the second bias vector, T represents the transposition operation, and T [1：K] It is a tensor composed of K matrices of dimension d×d, f represents the fusion function, h t T T [1：K] h s represents a K-dimensional tensor product;

[0021] Step 6: Represent the short text as r final Input the forward GRU model of the Bi-GRU model for processing and obtain the short text representation r final The forward hidden vector Then represent the short text r final The reverse order is input into the backward GRU model of the Bi-GRU model for processing, and the short text representation r is obtained. final The backward hidden vector

[0022] Step 7: Use formula (5) to get the short text representation r final The final hidden vector h:

[0023]

[0024] Step 8: Input the hidden vector h into the sigmoid layer, and use formula (6) to obtain the probability value y corresponding to the positive or negative emotion of the corresponding short text, so as to select the emotion corresponding to the larger probability value as the final emotion classification result of the corresponding short text:

[0025]

[0026] In formula (6), W3 is the third weight matrix, and b3 is the third bias vector.

[0027] The present invention provides an electronic device, comprising a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the short text implicit emotion classification method, and the processor is configured to execute the program stored in the memory.

[0028] The present invention provides a computer-readable storage medium having a computer program stored thereon. The computer-readable storage medium is characterized in that when the computer program is run by a processor, the steps of the short text implicit sentiment classification method are executed.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The present invention extracts fine-grained events described in short texts through event extraction, supplements the emotions of short texts that lack emotional words in the form of events, and then fuses the results of fine-grained event representation as enhanced information with the representation results of the short text itself to obtain short text representation enhanced with event information, which is then input into the Bi-GRU model to finally obtain the emotion classification result of the short text, thereby improving the accuracy of implicit emotion classification of short texts.

[0031] 2. This invention does not use coarse-grained event tuples consisting solely of the event subject, object, and predicate. Instead, it employs fine-grained event tuples, including the event type, event trigger, and other event elements. Compared to coarse-grained events, fine-grained events can better express the entire event described by the short text, thereby more accurately capturing the underlying sentiment within the short text. Furthermore, this invention employs an attention mechanism to characterize fine-grained events, assigning different weights to different elements within the fine-grained event. A weighted summation approach is then employed to derive the feature vector for the entire event tuple, fully accounting for the semantic characteristics of the short text and improving the accuracy of implicit sentiment classification for short texts.

[0032] 3. In the selection of classification model, the present invention adopts Bi-GRU neural network to solve the gradient explosion problem of traditional recurrent neural network. At the same time, it fully considers the context information within the short text representation vector and the context information that may exist between different short text representation vectors, effectively enhancing the representation effect of short text and improving the accuracy of short text implicit sentiment classification tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 The figure is an overall flow chart of the method of the present invention. DETAILED DESCRIPTION

[0034] In this embodiment, a short text implicit sentiment classification method based on fine-grained event information enhancement is to use the event extraction method to extract the fine-grained events described by the short text, and respectively characterize the events and short texts through the BERT pre-training model. First, the attention mechanism is used to calculate the weight score of each element in the fine-grained event, and the representation of the fine-grained event tuple is obtained by weighted summation. Then, the tensor combination method is used to fuse the event representation as enhanced information with the short text representation, thereby enhancing the representation effect of the short text. Finally, the Bi-GRU neural network is used to better learn the short text representation, thereby improving the accuracy of the short text implicit sentiment classification. Figure 1 Specifically, the method is performed in the following steps:

[0035] Step 1: Get the short texts in the comment data and build a data set. Perform data preprocessing on all the short texts in the data set to obtain the word segmentation set corresponding to each short text. The word segmentation set of any short text s = (s1, s2, ..., s i ,...,s a ), where s i represents the i-th word in the word segmentation set s, and a represents the total number of words in the word segmentation set s;

[0036] Step 2: Use the event extractor to extract single events from the word set s to extract the event type, trigger words and other event elements of the single event, thereby obtaining the fine-grained single event tuple t = (t1, t2, ..., t l ,...,t b ), where t l represents the lth element in the single event tuple t, b represents the total number of elements in the single event tuple t, l∈[1,b]; compared with the coarse-grained events that only include the event subject, predicate and object, the fine-grained events can more completely describe the entire event described in the short text, and the short text emotions contained in the coarse-grained events are also stronger.

[0037] Step 3: sort the single event tuple t into [cls]t1[sep]t2[sep]...[sep]t l [sep]...[sep]t b [sep] format is input into the BERT pre-trained model to obtain the hidden state of each event element in the single event tuple t, where the hidden state of the lth element is recorded as H l , [cls] represents the flag used to obtain the final representation result of the model, and [sep] represents the flag used to split each event element in the event tuple;

[0038] Step 4: Input the hidden state of each element in the single event tuple t into the attention mechanism network. First, assign different attention scores to all event elements in the single event tuple t according to their importance. Then, perform a weighted sum of the attention scores corresponding to each event element on all elements in the single event tuple t to obtain the representation vector of the entire single event tuple t.

[0039] Step 4.1: Use equations (1) and (2) to calculate the lth element t in the single event tuple t l The attention score α l :

[0040] u l =sigmoid(W1H l +b1) (1)

[0041]

[0042] In formula (1), W1 is the first weight matrix, b1 is the first bias vector, sigmoid represents the activation function, and u l is the lth element t in the single event tuple t l The new hidden state of

[0043] In formula (2), T represents the transposition operation, θ lRepresents the randomly initialized lth element t l The attention matrix;

[0044] Step 4.2: Use formula (3) to obtain the attention feature vector h of the single event tuple t t , and as the characterization vector of the single event tuple t:

[0045]

[0046] Step 5: Input the word segmentation set s into the BERT pre-training model to obtain the hidden vector h of the short text s , the relationship between event tuples and word sets is modeled by using a tensor combination method, and the interaction between single event tuple t and word set s is captured by bilinear tensor operation. The representation of fine-grained event tuples is used as enhanced information to improve the representation effect of the corresponding short text, so that the short text representation r enhanced by single event information is obtained using formula (4) final :

[0047]

[0048] In formula (4), W2 is the second weight matrix, b2 is the second bias vector, T represents the transposition operation, and T [1：K] It is a tensor composed of K matrices of dimension d×d, f represents the fusion function, h t T T [1：K] h s represents a K-dimensional tensor product;

[0049] Step 6: Use Bi-GRU model to model the short text representation enhanced by fine-grained event information, taking full account of the context information of the short text feature vector. final Input into the forward GRU model of the Bi-GRU model for processing and obtain the short text representation r final The forward hidden vector Then represent the short text r final The reverse order is input into the backward GRU model of the Bi-GRU model for processing, and the short text representation r is obtained. final The backward hidden vector

[0050] Step 7: Use formula (5) to get the short text representation r final The final hidden vector h:

[0051]

[0052] Step 8: Input the hidden vector h into the sigmoid layer, and use formula (6) to obtain the probability value y corresponding to the positive or negative sentiment of the corresponding short text, so as to select the sentiment corresponding to the larger probability value as the final sentiment classification result of the corresponding short text:

[0053]

[0054] In formula (6), W3 is the third weight matrix, and b3 is the third bias vector.

[0055] In this embodiment, an electronic device includes a memory and a processor. The memory is used to store a program that supports the processor to execute the above-mentioned short text implicit emotion classification method, and the processor is configured to execute the program stored in the memory.

[0056] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned short text implicit sentiment classification method are executed.

[0057] In summary, this method first extracts the fine-grained events described in the short text, uses the attention mechanism to obtain the representation of the fine-grained single event tuple, and then fuses the fine-grained event representation and the representation of the short text itself through tensor combination to obtain the short text representation with enhanced fine-grained event information. Since the emotions in the short text are mostly triggered by the events described in the short text, and the event elements in the fine-grained events will affect the emotional tendencies and intensity hidden in the short text, the method proposed in this invention can ultimately improve the accuracy of the short text implicit sentiment classification task.

Claims

1. A short text implicit sentiment classification method based on fine-grained event information enhancement, characterized by: The steps are as follows: Step 1: Get the short texts in the comment data and build a data set. Perform data preprocessing on all the short texts in the data set to obtain the word segmentation set corresponding to each short text. The word segmentation set of any short text s=(s1,s2,...,s i ,...,s a ), where s i Represents the i-th word in the word segmentation set s, and a represents the total number of words in the word segmentation set s; Step 2: Use the event extractor to extract single events from the word set s to extract the elements of the single event, thereby obtaining the fine-grained single event tuple t=(t1, t2, ..., t l ,...,t b ), where t l represents the lth element in the single event tuple t, b represents the total number of elements in the single event tuple t, l∈[1,b]; Step 3: Input the single event tuple t into the BERT pre-training model to obtain the hidden state of each element in the single event tuple t, where the hidden state of the lth element is recorded as H l ; Step 4: Input the hidden state of each element in the single event tuple t into the attention mechanism network to obtain the representation vector of the single event tuple t; Step 4.1: Use equations (1) and (2) to calculate the lth element t in the single event tuple t l The attention score α l : u l =sigmoid(W1H l +b1) (1) In formula (1), W1 is the first weight matrix, b1 is the first bias vector, sigmoid represents the activation function, and u l is the lth element t in the single event tuple t l The new hidden state of In formula (2), T represents the transposition operation, θ l Represents the randomly initialized lth element t l The attention matrix; Step 4.2: Use formula (3) to obtain the attention feature vector h of the single event tuple t t , and as the characterization vector of the single event tuple t: Step 5: Input the word segmentation set s into the BERT pre-training model to obtain the hidden vector h of the short text s , and thus use formula (4) to obtain the short text representation r of single event information enhancement final : In formula (4), W2 is the second weight matrix, b2 is the second bias vector, T represents the transposition operation, and T [1:K] It is a tensor composed of K matrices of dimension d×d, f represents the fusion function, h t T T [1:K] h s represents a K-dimensional tensor product; Step 6: Represent the short text as r final Input into the forward GRU model of the Bi-GRU model for processing and obtain the short text representation r final The forward hidden vector Then represent the short text r final The reverse order is input into the backward GRU model of the Bi-GRU model for processing, and the short text representation r is obtained. final The backward hidden vector Step 7: Use formula (5) to get the short text representation r final The final hidden vector h: Step 8: Input the hidden vector h into the sigmoid layer, and use formula (6) to obtain the probability value y corresponding to the positive or negative emotion of the corresponding short text, so as to select the emotion corresponding to the larger probability value as the final emotion classification result of the corresponding short text: In formula (6), W3 is the third weight matrix, and b3 is the third bias vector.

2. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the short text implicit sentiment classification method according to claim 1, and the processor is configured to execute the program stored in the memory.

3. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the short text implicit sentiment classification method according to claim 1 are executed.

Images