Short text sentiment classification method, medium and system based on public opinion big data analysis

Abstract

The present invention claims protection for a short text sentiment classification method, medium and system based on public opinion big data analysis, belonging to the field of text classification, the method comprising: converting the text and comment text vector set obtained by the vector acquisition model and the topic list obtained by the neural network multi-label classification model into an input sample in the format of single topic + text + comment text; constructing a capsule network model based on the self-topic attention mechanism as a sentiment classification model, designing the loss function of the model according to the topic weight, and using the above input sample as the input of the model; inputting the text to be predicted into the sentiment classification model for sentiment label prediction, and completing the sentiment classification of the comment short text. The text sentiment classification model is a capsule network with a self-topic attention mechanism added, which can capture the richer text features and corresponding sentiment labels in the comment short text vector according to different topic classifications, and more accurately extract the features of emotions in the short text text, and achieve high sentiment classification accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of text classification in natural language processing, and specifically relates to a sentiment classification method for short texts of university public opinion reviews based on a capsule network and self-semantic expansion and attention. Background Art

[0002] With the development of society and the advancement of the internet, a large number of netizens post text messages on social media platforms daily. As highly visible social institutions, universities are easily the focus of controversy when public opinion arises. Comments underlying public opinion implicitly reflect netizens' emotions regarding a particular issue. Therefore, in the current online environment, the ability to automatically and accurately identify netizens' emotions is crucial for maintaining the online environment, controlling university public opinion, and building a harmonious campus. Previous sentiment analysis focused on analyzing the body of public opinion, neglecting to effectively analyze the comments. Comments are typically short and concise, exhibiting varying semantic characteristics depending on the topic, yet they contain a wealth of key netizens' attitudes towards a specific public opinion event and are closely linked to the development of public opinion trends. Consequently, traditional text sentiment classification methods often struggle to accurately capture the sentiment of public opinion comments.

[0003] Previously, there were no methods for sentiment analysis of short texts in public opinion big data. This method extracts topics from the main text, then expands the public opinion commentary corpus by combining topics, text, and comments. By assigning different weights to different topics, a capsule network based on the self-topic attention mechanism is constructed to achieve sentiment classification. This method effectively solves the problem of sentiment classification in short texts from public opinion big data. Summary of the Invention

[0004] The present invention aims to solve the above problems in the prior art. It proposes a short text sentiment classification method, medium, and system based on public opinion big data analysis. The technical solution of the present invention is as follows:

[0005] A short text sentiment classification method based on public opinion big data analysis includes the following steps:

[0006] 101. Convert the text and comment text vector sets obtained by the vector acquisition model and the topic vector set obtained by the neural network multi-label classification model into input samples in the format of single topic + text + comment text; the vector acquisition model refers to any one of the vector space model VSM, the word frequency inverse text model TF / IDF, the word vector model word2vec, and the pre-trained model BERT; the neural network multi-label classification model refers to any one of the traditional recurrent neural network model RNN, the traditional time series model LSTM, and the traditional convolutional neural network model CNN.

[0007] 102. Construct a capsule network model based on the self-topic attention mechanism as a sentiment classification model for public opinion comments. Design the loss function of the model based on the topic weights and use the above input samples as the input of the model.

[0008] 103. Input the text to be predicted into the sentiment classification model to predict the sentiment label and complete the sentiment classification of the short comment text.

[0009] Furthermore, step 101 converts the text and comment text vector sets obtained by the vector acquisition model and the topic vector set obtained by the neural network multi-label classification model into input samples in the format of single topic + text + comment text, specifically:

[0010] The public opinion text and comment data are input into the vector acquisition model for preprocessing and encoding, so that each word in the original text has a corresponding feature representation in the vector space; each text is represented as X = {x1, x2, ..., x m}, where m represents the length of the body text, and the comment text is Y = {y1, y2, ..., y l}, where l represents the length of the comment text. For short text comments, the l value is usually smaller than the m value;

[0011] The pre-processed public opinion text is input into the neural network multi-label classification model to obtain the topic vector set of each corpus. Each topic text is represented by T = {t 11 , t 12 ,...t 1a , t 21 , t 22 ,...,t 2b , t 21 , t k1 ,...,t kq}, where k is the number of topics in a certain public opinion article, and a, b…q are the lengths of different topics in words;

[0012] Focusing on the topic by averaging, the sample vector is represented as X * ={∑(t1, ...t 1a ) / a,x1,x2,...x m ,y1,y2,...,y l}, where q is the number of topics, m is the length of the text, and l is the length of the comment text. For a comment text, the number of input samples is n, and n represents the number of topics to which the text belongs.

[0013] Furthermore, the step 102 constructs a capsule network model based on the self-topic attention mechanism as a sentiment classification model for public opinion comments, designs a loss function for the model based on the topic weight, and uses the above input sample as the input of the model, specifically:

[0014] A capsule network model based on the self-topic attention mechanism is constructed, which divides the input capsule into two parts: the topic and the other. Dynamic routing is iterated by learning a matrix based on the viewpoint-invariant relationship between the part and the whole, which represents the self-topic attention. The input capsule is multiplied by the learning matrix of the dynamic routing as the self-feature extraction layer, and the capsule vector υ and the corresponding routing parameter b are calculated and updated:

[0015]

[0016]

[0017]

[0018] Where r is the number of capsule network layers, b i is the attention weight parameter of the i-th vector, initially a random initial matrix, q i , k i ,υ i is the attention parameter, which is determined by the dot product of the input capsule and parameter b, a r Calculate parameters for the capsule network original routing.

[0019] Then, the output capsules υ are connected in series and input into the feed-forward network FFN, which consists of three linear transformations, where the activation function is Sigmoid.

[0020]

[0021] x represents the text vector, b1, b2, b3, W1, W2, and W3 represent the weight parameters and parameter matrices respectively, which are randomly initialized and iteratively trained by the feedback neural network.

[0022] The input capsule of the next layer is obtained through the compression function. The dimension of the output is not changed by squeezing, only the length is changed:

[0023]

[0024] Furthermore, the loss function of the model is designed according to the topic weight, specifically:

[0025] The cross entropy loss function is used as the sentiment judgment task, and the corresponding weight is increased according to the topic:

[0026]

[0027] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the short text sentiment classification method based on public opinion big data analysis as described in any one of the items.

[0028] A multi-label text classification system based on public opinion big data analysis, comprising:

[0029] Encoding module: used to input the public opinion text and comment data into the vector acquisition model respectively and encode them into the public opinion text vector set and comment data text vector set; Topic vector set acquisition module: used to input the public opinion text vector set into the LSTM to build a multi-label topic classification model and obtain the topic vector set;

[0030] Conversion module: used to convert the public opinion text and comment text and the corresponding topic list into input samples in the format of single topic + text + comment text;

[0031] Sentiment classification model: used to build a capsule network model based on the attention mechanism as a sentiment classification model for public opinion comments; the above input samples are used as the input of the capsule network model;

[0032] Prediction module: used to input the text to be predicted into the capsule network model to predict the sentiment label and complete the sentiment classification of the short comment text.

[0033] The advantages and beneficial effects of the present invention are as follows:

[0034] The present invention adopts self-corpus expansion and attention-based capsule network to solve the short text sentiment classification problem of university public opinion comments. By analyzing the theme of the public opinion text, the short text comments are re-encoded to give them thematic characteristics, which is conducive to sentiment analysis. Secondly, through the capsule network based on the attention mechanism, the text connotation is analyzed to add richer context-related corpus information to the model training. In the past, there has been no short text sentiment analysis method for public opinion big data. This method extracts the theme of the text content, and then expands the short text corpus information of public opinion comments by combining the theme, text and comments, and different weights for different topics. A capsule network based on the self-topic attention mechanism is constructed to achieve sentiment classification, thereby greatly improving the accuracy of sentiment classification of short text information data such as university public opinion comments.

[0035] The innovations are:

[0036] 1. For short texts of public opinion, we constructed the topic + text + comments as input samples

[0037] 2. Improvement of the original capsule network dynamic routing, incorporating a multi-head self-attention mechanism

[0038] The loss function is improved to associate the comment sentiment prediction results with the importance of the topic. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is an overall flow chart of a preferred embodiment of the present invention;

[0040] Figure 2 An example diagram of converting the university public opinion text information description and the corresponding topic list into an input sample of the present invention;

[0041] Figure 3 This is a network structure diagram of the present invention. DETAILED DESCRIPTION

[0042] The following will describe the technical solutions in the embodiments of the present invention in detail with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of the present invention.

[0043] The technical solution of the present invention to solve the above technical problems is:

[0044] like Figure 1 As shown, a capsule network-based method for sentiment classification of university public opinion comments in short texts is presented, based on self-semantic expansion and multi-head attention. The method includes: encoding the public opinion text and comment data into a vector acquisition model into a text vector set; inputting the text vector set into an LSTM to construct a multi-label topic classification model to obtain a topic vector set; converting the text and comment text and their corresponding topic lists into input samples in the format of a single topic + text + comment text; inputting the input samples into a text sentiment classification model to obtain a classification model; and inputting the text to be predicted into the model to predict the sentiment label, thereby completing the sentiment classification of the short text.

[0045] A sentiment classification method for short texts of university public opinion reviews based on capsule networks with self-semantic expansion and attention includes:

[0046] 101. Convert the text and comment text vector sets obtained by the vector acquisition model and the topic vector set obtained by the neural network multi-label classification model into input samples in the format of single topic + text + comment text; the vector acquisition model refers to any one of the vector space model VSM, the word frequency inverse text model TF / IDF, the word vector model word2vec, and the pre-trained model BERT (Chinese); the neural network multi-label classification model refers to any one of the traditional recurrent neural network model RNN, the traditional time series model LSTM, and the traditional convolutional neural network model CNN.

[0047] 102. Construct a capsule network model based on the self-topic attention mechanism as a sentiment classification model for public opinion comments. Design the loss function of the model based on the topic weights and use the above input samples as the input of the model.

[0048] 103. Input the text to be predicted into the sentiment classification model to predict the sentiment label and complete the sentiment classification of the short comment text.

[0049] Furthermore, step 101 converts the text and comment text vector sets obtained by the vector acquisition model and the topic vector set obtained by the neural network multi-label classification model into input samples in the format of single topic + text + comment text, specifically:

[0050] The public opinion text and comment data are input into the vector acquisition model for preprocessing and encoding, so that each word in the original text has a corresponding feature representation in the vector space; each text is represented as X = {x1, x2, ..., x m}, where m represents the length of the body text, and the comment text is Y = {y1, y2, ..., y l}, where l represents the length of the comment text. For short text comments, the l value is usually smaller than the m value;

[0051] The pre-processed public opinion text is input into the neural network multi-label classification model to obtain the topic vector set of each corpus. Each topic text is represented by T = {t 11 , t 12 ,...t 1a , t 21 , t 22 ,...,t 2b , t 21 , t k1 ,...,t kq}, where k is the number of topics in a certain public opinion article, and a, b…q are the lengths of different topics in words;

[0052] Focusing on the topic by averaging, the sample vector is represented as X * ={∑(t1, ...t 1a ) / a,x1,x2,...x m ,y1,y2,...,y l}, where q is the number of topics, m is the length of the text, and l is the length of the comment text. For a comment text, the number of input samples is n, and n represents the number of topics to which the text belongs.

[0053] Furthermore, the step 102 constructs a capsule network model based on the self-topic attention mechanism as a sentiment classification model for public opinion comments, designs a loss function for the model based on the topic weight, and uses the above input sample as the input of the model, specifically:

[0054] A capsule network model based on the self-topic attention mechanism is constructed, which divides the input capsule into two parts: the topic and the other. Dynamic routing is iterated by learning a matrix based on the viewpoint-invariant relationship between the part and the whole, which represents the self-topic attention. The input capsule is multiplied by the learning matrix of the dynamic routing as the self-feature extraction layer, and the capsule vector υ and the corresponding routing parameter b are calculated and updated:

[0055]

[0056]

[0057]

[0058] Where r is the number of capsule network layers, b i is the attention weight parameter of the i-th vector, initially a random initial matrix, q i , k i ,υ i is the attention parameter, which is determined by the dot product of the input capsule and parameter b, a r Calculate parameters for the capsule network original routing.

[0059] Then, the output capsules υ are connected in series and input into the feed-forward network FFN, which consists of three linear transformations, where the activation function is Sigmoid.

[0060]

[0061] x represents the text vector, b1, b2, b3, W1, W2, and W3 represent the weight parameters and parameter matrices respectively, which are randomly initialized and iteratively trained by the feedback neural network.

[0062] The input capsule of the next layer is obtained through the compression function. The dimension of the output is not changed by squeezing, only the length is changed:

[0063]

[0064] Furthermore, the loss function of the model is designed according to the topic weight, specifically:

[0065] The cross entropy loss function is used as the sentiment judgment task, and the corresponding weight is increased according to the topic:

[0066]

[0067] Thereafter, the text to be predicted is input into the sentiment classification model for sentiment label prediction to complete the sentiment classification of the short comment text.

[0068] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0069] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0070] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0071] The above embodiments should be understood as merely illustrating the present invention and not as limiting the scope of protection of the present invention. After reading the contents of the present invention, technicians may make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A short text sentiment classification method based on public opinion big data analysis, characterized in that: The following steps are involved:

101. Convert the text and comment text vector sets obtained by the vector acquisition model and the topic vector set obtained by the neural network multi-label classification model into input samples in the format of single topic + text + comment text; the vector acquisition model refers to any one of the vector space model VSM, the word frequency inverse text model TF / IDF, the word vector model word2vec, and the pre-trained model BERT; the neural network multi-label classification model refers to any one of the traditional recurrent neural network model RNN, the traditional time series model LSTM, and the traditional convolutional neural network model CNN; 102. Construct a capsule network model based on the self-topic attention mechanism as a sentiment classification model for public opinion comments, design the loss function of the model according to the topic weight, and use the above input samples as the input of the model; 103. Input the text to be predicted into the sentiment classification model to predict sentiment labels, and complete sentiment classification of the short comment text; The step 101 converts the text and comment text vector sets obtained by the vector acquisition model and the topic vector set obtained by the neural network multi-label classification model into input samples in the format of single topic + text + comment text, specifically: The public opinion text and comment data are input into the vector acquisition model for preprocessing and encoding, so that each word in the original text has a corresponding feature representation in the vector space; each text is represented as X = {x1, x2, ..., x m }, where m represents the length of the main text, and the comment text is Y = {y1, y2, ..., y l }, where l represents the length of the comment text. For short text comments, the l value is smaller than the m value; The preprocessed public opinion text is input into the neural network multi-label classification model to obtain the topic vector set of each corpus. Each topic text is represented by T = {t 11 , t 12 ,...t 1a , t 21 , t 22 , ..., t 2b , t 21 , t k1 , ..., t kq }, where k is the number of topics in a public opinion article, and a, b…q are the lengths of different topics in words; Focusing on the topic by averaging, the sample vector is represented as X * ={∑(t1, ...t 1a ) / a,x1,x2,...x m , y1, y2, ..., y l }, where q is the number of topics, m is the length of the text, and l is the length of the comment text; for a comment text, the number of input samples is n, and n represents the number of topics to which the text belongs; The step 102 constructs a capsule network model based on the self-topic attention mechanism as a sentiment classification model for public opinion comments, designs a loss function of the model according to the topic weight, and uses the above input sample as the input of the model, specifically: Construct a capsule network model based on the self-topic attention mechanism, and divide the input capsule into two parts: the topic and the other. Iterate the dynamic routing through the learning matrix based on the viewpoint-invariant relationship between the part and the whole that represents the self-topic attention. Multiply the input capsule with the learning result matrix of the dynamic routing as the self-feature extraction layer, and calculate and update the capsule vector υ and the corresponding routing b parameter: Where r is the number of capsule network layers, b i is the attention weight parameter of the i-th vector, initially a random initial matrix, q i , k i , i is the attention parameter, which is determined by the dot product of the input capsule and parameter b, a r Calculate parameters for the capsule network original routing; Then, the output capsules υ are connected in series and input into the feed-forward network FFN, which consists of three linear transformations, where the activation function is Sigmod, x represents the text vector, b1, b2, b3, W1, W2, W3 represent the weight parameter and parameter matrix respectively, which are randomly initialized and iteratively trained by the feedback neural network; The input capsule of the next layer is obtained through the compression function. The dimension of the output is not changed by squeezing, only the length is changed: The loss function of the model is designed according to the topic weight, specifically: The cross entropy loss function is used as the sentiment judgment task, and the corresponding weight is increased according to the topic: Afterwards, the text to be predicted is input into the sentiment classification model to perform sentiment label prediction, thereby completing sentiment classification of the short comment text.

2. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the short text sentiment classification method based on public opinion big data analysis as described in claim 1.

3. A multi-label text classification system based on public opinion big data analysis, characterized in that: include: Encoding and conversion module: converts the text and comment text vector sets obtained by the vector acquisition model and the topic vector set obtained by the neural network multi-label classification model into input samples in the format of single topic + text + comment text; Sentiment classification model: Construct a capsule network model based on the self-topic attention mechanism as the sentiment classification model for public opinion comments, design the loss function of the model according to the topic weight, and use the above input samples as the input of the capsule network model; Prediction module: used to input the text to be predicted into the capsule network model to predict the sentiment label and complete the sentiment classification of the short comment text; The text and comment text vector sets obtained by the vector acquisition model and the topic vector set obtained by the neural network multi-label classification model are converted into input samples in the format of single topic + text + comment text, specifically: The public opinion text and comment data are input into the vector acquisition model for preprocessing and encoding, so that each word in the original text has a corresponding feature representation in the vector space; each text is represented as X = {x1, x2, ..., x m }, where m represents the length of the main text, and the comment text is Y = {y1, y2, ..., y l }, where l represents the length of the comment text. For short text comments, the l value is smaller than the m value; The preprocessed public opinion text is input into the neural network multi-label classification model to obtain the topic vector set of each corpus. Each topic text is represented by T = {t 11 , t 12 ,...t 1a , t 21 , t 22 , ..., t 2b , t 21 , t k1 , ..., t kq }, where k is the number of topics in a public opinion article, and a, b…q are the lengths of different topics in words; Focusing on the topic by averaging, the sample vector is represented as X * ={∑(t1, ...t 1a ) / a,x1,x2,...x m , y1, y2, ..., y l }, where q is the number of topics, m is the length of the text, and l is the length of the comment text; for a comment text, the number of input samples is n, and n represents the number of topics to which the text belongs; The capsule network model based on the self-topic attention mechanism is constructed as the sentiment classification model of public opinion comments, the loss function of the model is designed according to the topic weight, and the above input samples are used as the input of the model, specifically: Construct a capsule network model based on the self-topic attention mechanism, and divide the input capsule into two parts: the topic and the other. Iterate the dynamic routing through the learning matrix based on the viewpoint-invariant relationship between the part and the whole that represents the self-topic attention. Multiply the input capsule with the learning result matrix of the dynamic routing as the self-feature extraction layer, and calculate and update the capsule vector υ and the corresponding routing b parameter: Where r is the number of capsule network layers, b i is the attention weight parameter of the i-th vector, initially a random initial matrix, q i , k i , i is the attention parameter, which is determined by the dot product of the input capsule and parameter b, a r Calculate parameters for the capsule network original routing; Then, the output capsules υ are connected in series and input into the feed-forward network FFN, which consists of three linear transformations, where the activation function is Sigmod, x represents the text vector, b1, b2, b3, W1, W2, W3 represent the weight parameter and parameter matrix respectively, which are randomly initialized and iteratively trained by the feedback neural network; The input capsule of the next layer is obtained through the compression function. The dimension of the output is not changed by squeezing, only the length is changed: The loss function of the model is designed according to the topic weight, specifically: The cross entropy loss function is used as the sentiment judgment task, and the corresponding weight is increased according to the topic: Afterwards, the text to be predicted is input into the sentiment classification model to perform sentiment label prediction, thereby completing sentiment classification of the short comment text.

Images