Tobacco Sentiment Analysis Method, Device and Medium

Abstract

The present invention provides a tobacco sentiment analysis method, device, and medium. The method comprises constructing a tobacco sentiment analysis model; training the tobacco sentiment analysis model; and inputting pre-analyzed text into the trained tobacco sentiment analysis model to obtain the sentiment polarity of the text, where the sentiment polarity includes positive, neutral, and negative. The present invention can accurately and effectively predict the sentiment polarity of user reviews in the tobacco field.

Description

Technical Field

[0001] The present invention relates to the field of tobacco technology, and in particular to a tobacco sentiment analysis method, device and medium. Background Art

[0002] Sentiment analysis is the process of mining the emotional tendencies of information. It is primarily used for public opinion monitoring, product review analysis, and information retrieval. As more and more users express their opinions and experiences on various social platforms, a vast amount of emotionally charged commentary text data is generated, which is extremely valuable. In recent years, sentiment analysis of product reviews has garnered significant attention.

[0003] Currently, mainstream sentiment analysis methods are based on existing public datasets, such as SemEval-2014 task 4 and Twitter. Both are English datasets. Their focus is primarily on food and entertainment, not tobacco. Currently, demand for tobacco consumption is rapidly increasing. From 2000 to 2019, domestic tobacco consumption grew at an average annual rate of 11.26%. Tobacco is a unique and vast field, yet research on it is virtually nonexistent.

[0004] There are three main types of research methods for sentiment analysis:

[0005] The first approach is based on sentiment dictionaries. This method requires constructing a domain-specific sentiment lexicon and combining it with manually designed rules to determine sentiment polarity. However, due to the constant emergence of new words on the internet, sentiment dictionaries cannot be updated in real time, resulting in a lack of accuracy in sentiment analysis results. The meaning and importance of the same word can vary across different domains, and basic sentiment lexicon analysis cannot account for these differences in terms, leading to inaccurate analysis results.

[0006] The second approach is machine learning. This method primarily trains classifiers, such as Naive Bayes, Maximum Entropy, Support Vector Machines (SVM), and Logistic Regression. Compared to methods based on sentiment lexicons, machine learning offers significantly improved performance. However, it requires high feature engineering and is not well adapted to diverse tasks in different fields.

[0007] The third approach is deep learning. It primarily focuses on two types of neural networks: convolutional neural networks (CNNs) and recurrent neural networks (RNNs). Training datasets require large amounts of labeled data. Compared to machine learning, deep learning approaches are more domain-specific. However, due to the lack of tobacco datasets for training, their effectiveness in the tobacco field has been relatively limited.

[0008] There are three main difficulties that lead to research difficulties in the field of tobacco consumption:

[0009] The first is that there are currently no publicly available tobacco datasets available for research. Tobacco is a specialized field, and research based on public datasets from other fields is not fully applicable to the tobacco field. This lack of datasets hinders further research.

[0010] The second difficulty is that online reviews are mostly diverse and informal. Directly scraping data will contain a large amount of duplicate data, typos, irrelevant content, and other invalid information.

[0011] The third difficulty is that a single review sentence may contain multiple aspects of evaluation, each with different sentiment. Although aspect-based sentiment analysis can identify the sentiment polarity of specific aspects within a sentence, existing methods cannot determine the overall sentiment polarity of a sentence. For example, the sentence "The packaging is beautiful, but the taste is terrible, the fragrance is not good, and it's not worth the price" has four aspects: "packaging," "taste," "fragrance," and "price," which are associated with negative and positive sentiment, respectively. While "packaging" is positive, other aspects such as "taste," "fragrance," and "price" are negative.

[0012] In summary, existing research methods cannot overcome the difficulties of tobacco datasets and cannot achieve good results in the tobacco field. Summary of the Invention

[0013] In view of one or more problems existing in the prior art, the present invention provides a tobacco sentiment analysis method, comprising:

[0014] Step S1, constructing a tobacco sentiment analysis model (EHCRNN model, based on the model framework of Ernie, HAN, CNN, and RNN networks);

[0015] Step S2, training the tobacco sentiment analysis model;

[0016] Step S3, inputting the pre-analyzed text into the trained tobacco sentiment analysis model to obtain the sentiment polarity of the text, where the sentiment polarity includes positive, neutral, and negative;

[0017] Wherein, the step S1 includes:

[0018] Step S11: introduce the multi-head attention mechanism and use the pre-trained model Ernie to obtain the word vector matrix;

[0019] Step S12: Using the word vector matrix as input to a convolutional neural network (CNN), extracting a first eigenvector matrix, using the first eigenvector matrix as input to a bidirectional long short-term memory network (BiLSTM, based on RNN) to obtain semantic features and contextual information, and embedding the semantic features and contextual information into the first eigenvector matrix to obtain a second eigenvector matrix;

[0020] Step S13, obtaining the semantic-level attention and word-level attention of the second eigenvector matrix through a hierarchical attention network (HAN), and obtaining a third eigenvector matrix through a weighted combination of the second eigenvector matrix, the semantic-level attention, and the word-level attention;

[0021] Step S14: predict the sentiment polarity of the third eigenvector matrix using a softmax function.

[0022] The present invention introduces a multi-head attention mechanism into the word vector matrix, and also introduces attention through a hierarchical attention network after CNN and BiLSTM, thereby realizing the fusion and expansion of CNN and BiLSTM and improving the accuracy and speed of feature extraction.

[0023] According to one aspect of the present invention, step S11 includes:

[0024] Mark the sentences in the text to form a text sequence. The tags include [CLS] and [SEP]. [CLS] means it is placed at the beginning of a sentence, and [SEP] is used to separate two input sentences. Preferably, the text sequence is , n represents the length of the sentence, For the first words;

[0025] Embed the original words, fragments and sequence positions into the above text sequence;

[0026] Multi-head attention is introduced to obtain the correlation and importance between different words in different sentences in the embedded text sequence. Preferably, the correlation and importance between different words in different sentences are obtained by the following formula:

[0027]

[0028] Among them, Q (Query), K (Key), V (Value) is the input word vector matrix, is the input word vector dimension, For each word's relevance to other words, is the weighted sum of word vectors, is the importance of different words in different sentences, the larger the weighted sum, the more important it is; is the attention operation, r is the head representation in the multi-head attention;

[0029] The latent variable matrix output by Ernie is used as the word vector matrix. Preferably, the latent variable matrix is , where h represents the latent variable dimension of Ernie.

[0030] This paper utilizes Ernie, a dynamic pre-trained language model, to generate semantic feature information for words based on context. Ernie directly models prior semantic knowledge units, enhancing the model's semantic representation capabilities. Incorporating Ernie into model design to generate semantic feature information effectively addresses the polysemy problem of textual vocabulary in the tobacco field, thereby improving the accuracy of identifying sensitive information in tobacco-related textual vocabulary.

[0031] According to one aspect of the present invention, step S12 includes:

[0032] The word vector output by Ernie is used as the input of the convolutional neural network (CNN) to extract features and obtain the first eigenvector matrix. Preferably, convolution kernels of different sizes are used to extract features. More preferably, the convolution kernels of different sizes are 2, 3 and 4 respectively. Preferably, the nonlinear activation function ReLU is used between the layers of the convolutional neural network (CNN). Preferably, the first eigenvector matrix is obtained by the following formula:

[0033]

[0034] in, is the weight term, Represents the word vector matrix Go to The word vector of the row, is the bias term, is the first eigenvector matrix feature vectors;

[0035] Perform maximum pooling on the first eigenvector matrix. Preferably, perform maximum pooling on the first eigenvector matrix using the following formula:

[0036]

[0037] in, is the eigenvector of the first eigenvector matrix after pooling;

[0038] The first eigenvector matrix after maximum pooling is used as the input of a bidirectional long short-term memory network (BiLSTM) to obtain semantic features and context information, and the semantic features and context information are embedded into the first eigenvector matrix after maximum pooling to obtain a second eigenvector matrix.

[0039] This invention uses CNNs and LSTMs to deeply extract sentence-level word vector representations, further abstracting the representations enhanced by Ernie, extracting and abstracting contextual semantic features, and improving feature expressiveness. LSTMs effectively address the vanishing and exploding gradient problems of RNNs. The BiLSTM in this invention consists of two LSTM networks: a forward LSTM that collects and utilizes past information, and a reverse LSTM that leverages future information. This allows, at time t, to utilize both information from time t-1 and time t+1. Therefore, BiLSTM captures more information, enabling better prediction and classification of sequence data.

[0040] According to one aspect of the present invention, step S13 includes:

[0041] The third eigenvector matrix is obtained by the following formula:

[0042]

[0043]

[0044]

[0045] in, is the second eigenvector matrix, For the The vector representation of a sentence, is the bias term, is a randomly initialized vector, is the importance weight, The obtained weighted vector representation of the 𝑖th sentence based on weights is also the 𝑖th eigenvector of the third eigenvector matrix.

[0046] The present invention utilizes a Hierarchical Attention Network (HAN) to simultaneously leverage semantic-level attention and word-level attention, learning the importance of sentences and words simultaneously and obtaining the final weight distribution through corresponding aggregation operations. Therefore, the present invention introduces a Hierarchical Attention Network (HAN) to distribute weights at both the vocabulary and sentence levels, capturing more important information and significantly enhancing the model's text feature extraction capabilities and improving text classification and recognition accuracy.

[0047] According to one aspect of the present invention, step S14 includes:

[0048] Use softmax to predict the sentiment polarity of the third eigenvector matrix by the following formula:

[0049]

[0050] Wherein, M is the number of eigenvectors of the third eigenvector matrix.

[0051] The present invention combines softmax to give classification prediction results based on probability, thereby achieving the effect of text sentiment analysis.

[0052] According to one aspect of the present invention, step S2 further includes constructing a tobacco dataset.

[0053] According to one aspect of the present invention, the step of constructing a tobacco dataset comprises:

[0054] Collect multiple comment texts about tobacco;

[0055] Annotating the plurality of review texts using different methods, including sentiment dictionaries and SnowNLP;

[0056] Filter out annotations that are consistent in different ways.

[0057] In the field of artificial intelligence, machine learning and deep learning models rely on extensive annotations in datasets for training. However, producing large numbers of labeled training samples is extremely difficult, labor-intensive, and time-consuming. In reality, there is often only a small amount of labeled data and a large amount of unlabeled data. This invention can save significant labor costs and rapidly construct datasets in new fields, thereby providing a basis for further research.

[0058] According to one aspect of the present invention, the step of constructing a tobacco dataset further comprises:

[0059] Cross-validate the annotations that are inconsistent.

[0060] According to one aspect of the present invention, step S2 includes:

[0061] The sum of cross entropy loss and L2 regularization loss is used as the loss function;

[0062] Input tobacco dataset into tobacco sentiment analysis model for training;

[0063] The tobacco sentiment analysis model corresponding to the minimum loss function is used as the trained tobacco sentiment analysis model.

[0064] According to one aspect of the present invention, it also includes:

[0065] The tobacco dataset is divided into training set, validation set and test set;

[0066] The tobacco sentiment analysis model is trained using the above training set;

[0067] Parameter tuning through validation set;

[0068] The tobacco sentiment analysis model after parameter tuning is evaluated using a test set, where the evaluation indicators include one or more of accuracy, precision, recall rate, and f1-score.

[0069] According to one aspect of the present invention, the step of annotating the plurality of comment texts in different ways includes:

[0070] Establishing a database, wherein the database includes a basic sentiment dictionary, a degree adverb dictionary, a conjunction dictionary, a negation word dictionary, a stop word dictionary and a tobacco sentiment dictionary;

[0071] Segment the review text into sentences and words, and obtain each sentence in the review text, each clause in each sentence, and each word in each sentence and each clause. Preferably, use Jieba to segment the clauses.

[0072] Analyze the modification relationship between negative words and sentiment words in sentences through the database;

[0073] Analyze the tendency value of each sentence through the database;

[0074] Accumulate the sentiment value of each sentence to obtain the positive sentiment tendency score and negative sentiment tendency score of the comment text;

[0075] The Sigmoid function is used to obtain the sentiment tendency, positive sentiment tendency degree and negative sentiment tendency degree of the review text.

[0076] The results obtained by machine learning models without training on domain-specific datasets will be limited in domain adaptability. The above method uses dictionaries and rules to perform sentiment analysis, and can produce sentiment analysis results that are suitable for specific domains without annotations.

[0077] According to one aspect of the present invention, the step of establishing a database includes:

[0078] Use LDA topic model to select the topic of the review text;

[0079] Initialize the attribute word dictionary according to the above topics;

[0080] Use word2vec to obtain the distance between word vectors in the attribute word dictionary after initialization;

[0081] Classify the word vectors whose distance is greater than a set value into topics and construct a tobacco sentiment dictionary. Preferably, the set value is 0.8;

[0082] Build a basic sentiment dictionary, a degree adverb dictionary, a conjunction dictionary, a negation word dictionary, and a stop word dictionary.

[0083] Preferably, the HowNet sentiment dictionary is used as the basic sentiment dictionary.

[0084] Prioritize the use of the Harbin Institute of Technology stop word dictionary as the stop word dictionary

[0085] Preferably, the conjunction word list and the negation word list on the Internet are used to organize the conjunction word dictionary and the negation word dictionary through data processing.

[0086] Preferably, a degree adverb dictionary is constructed based on HowNet degree level words, and weights are assigned according to extreme, high, medium-high, medium and low to obtain a degree adverb dictionary.

[0087] According to one aspect of the present invention, the step of analyzing the modification relationship between negative words and sentiment words in a sentence through a database includes:

[0088] Using the tobacco sentiment dictionary, degree adverb dictionary, conjunction dictionary, and negation dictionary, we identify sentiment words, degree adverbs, negation words, and transitional conjunctions in the review texts and embed the positions of sentiment words, degree adverbs, negation words, and transitional conjunctions.

[0089] Get every adverb of degree and every negation;

[0090] The sentiment word closest to each degree adverb is used as the modification target of the degree adverb;

[0091] The sentiment word closest to each negative word is used as the modification target of the negative word;

[0092] The modification relationship is determined according to the number of negation words. When the number of times a negation word appears in a sentence is odd, the modification relationship of the negation word to the sentiment word is negative; when the number of times a negation word appears in a sentence is even, the modification relationship of the negation word to the sentiment word is positive.

[0093] The present invention utilizes the above rules to match Chinese text usage and improves the accuracy of the sentiment dictionary analysis model.

[0094] According to one aspect of the present invention, the step of analyzing the tendency value of each sentence through the database includes:

[0095] The sentiment score of each clause is obtained by the following formula

[0096]

[0097] in, Indicates the modification relationship between negative words and sentiment words. The score of the adverb of degree, 、 Represent the sentiment scores of sentiment words and clauses respectively, Indicates the number of transitional conjunctions;

[0098] When a sentence includes multiple clauses, and both the clauses with positive sentiment scores and the clauses with negative sentiment scores are not 0, reduce the positive sentiment score and / or increase the negative sentiment score;

[0099] When the transitional conjunction is not 0, the sentiment score of the clause after the transitional conjunction is increased and / or the sentiment score of the clause before the transitional conjunction is decreased.

[0100] According to one aspect of the present invention, the step of using the Sigmoid function to obtain the emotional tendency, positive emotional tendency degree, and negative emotional tendency degree of the comment text includes:

[0101] The emotional tendency, positive emotional tendency and negative emotional tendency of the comment text are obtained by the following formula

[0102]

[0103]

[0104] in, Represents the positive sentiment score of the comment text, Indicates the negative sentiment score of the comment text, Indicates the percentage of negative sentiment scores, Indicates the percentage of positive sentiment score. The larger the percentage, the greater the degree of sentiment tendency.

[0105] According to a second aspect of the present invention, a tobacco sentiment analysis system is provided, comprising:

[0106] Model building department, building tobacco sentiment analysis model;

[0107] The training department trains the tobacco sentiment analysis model built by the model building department;

[0108] The prediction unit inputs the pre-analyzed text into the tobacco sentiment analysis model trained by the training unit to obtain the sentiment polarity of the text, where the sentiment polarity includes positive, neutral, and negative;

[0109] Among them, the model construction part includes an embedding layer, a feature extraction layer, a hierarchical attention network and a sentiment polarity analysis model. The embedding layer introduces a multi-head attention mechanism and uses the pre-trained model Ernie to obtain a word vector matrix; the feature extraction layer includes a convolutional neural network (CNN) and a bidirectional long short-term memory network (BiLSTM). The word vector matrix of the embedding layer is the input of the convolutional neural network (CNN). The convolutional neural network extracts a first eigenvector matrix, which is the input of the bidirectional long short-term memory network (BiLSTM). The bidirectional long short-term memory network obtains semantic features and contextual information, and embeds the semantic features and contextual information into the first eigenvector matrix to obtain a second eigenvector matrix; the hierarchical attention network extracts the semantic-level attention and word-level attention of the second eigenvector matrix output by the feature extraction layer, and obtains a third eigenvector matrix by weighted combination of the second eigenvector matrix, the semantic-level attention and the word-level attention; the sentiment polarity analysis model predicts the sentiment polarity of the third eigenvector matrix output by the hierarchical attention network through a softmax function.

[0110] According to a third aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a tobacco sentiment analysis program, and when the tobacco sentiment analysis program is executed by the processor, the steps of the tobacco sentiment analysis method described above are implemented.

[0111] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a tobacco sentiment analysis program. When the tobacco sentiment analysis program is executed by a processor, the steps of the above-mentioned tobacco sentiment analysis method are implemented.

[0112] The LSTM (Long Short-Term Memory) model proposed by Hochreiter et al. can receive and process sentence sequences of different lengths and capture the text features therein, but is not suitable for parallel computing.

[0113] The CNN model proposed by Kim can also be used for sentiment analysis. CNN supports parallel computing more, but its local and entity negative correlations will affect the model. After several layers of pooling, the positional features of words will also be lost, and the prediction results will be adversely affected.

[0114] The Ernie model uses a large dataset for pre-training in unsupervised learning. Post-training, it is fine-tuned using specific datasets to adapt to specific scenarios. In many scenarios, Ernie achieves better results than LSTM and CNN models. However, it is a graph-based mechanism and not suitable for knowledge-driven NLP tasks. Experiments have shown that it is inferior to knowledge-enhanced network models in sentiment analysis in the tobacco field.

[0115] In summary, the existing technology mainly uses text features to extract information, without considering issues such as the identification of semantic knowledge entities in the text, the extraction of context-related text features, and text semantic reasoning. Therefore, for the problem of sentiment analysis in the tobacco field, several methods in the existing technology are not accurate enough.

[0116] The tobacco sentiment analysis system of the present invention combines the latest advances in natural language processing, including an embedding layer, a feature extraction layer, a hierarchical attention network, and a sentiment polarity analysis model. The tobacco sentiment analysis method of the present invention uses a pre-trained Ernie model to embed input. Feature extraction primarily involves convolution and pooling of a convolutional neural network (CNN) and a bidirectional long short-term memory network (BiLSTM), effectively capturing and utilizing contextual information. A hierarchical attention network (HAN) focuses on both word and sentence levels, enhancing the model's text feature extraction capabilities. A softmax function is used to predict the sentiment polarity of sentences. Taking into account the identification of semantic knowledge entities in the text, the extraction of contextual text features, and text semantic reasoning, the method can accurately and effectively predict the sentiment polarity of user reviews in the tobacco field.

[0117] The present invention not only verifies the EHCRNN model of the present invention on a new tobacco dataset, but also selects the public NLPCC2014 task dataset for experimental verification. The experiments show that the present invention outperforms the existing models on both the tobacco dataset and the public dataset.

[0118] The present invention also constructs a complete tobacco data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0119] Figure 1 This is a flow chart of an embodiment of the tobacco sentiment analysis method of the present invention;

[0120] Figure 2 This is a flow chart of an embodiment of the method for constructing a tobacco sentiment analysis model according to the present invention;

[0121] Figure 3 is a flow chart of an embodiment of the method for constructing a tobacco dataset according to the present invention;

[0122] Figure 4 This is a flow chart of an embodiment of the method for annotating multiple review texts using a sentiment dictionary according to the present invention;

[0123] Figure 5 is a schematic diagram of a structural block diagram of an embodiment of the tobacco sentiment analysis system of the present invention;

[0124] Figure 6 It is a coordinate graph of input size and accuracy. DETAILED DESCRIPTION

[0125] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and description are to be considered as illustrative in nature and not restrictive.

[0126] Figure 1 FIG. 1 is a flow chart of an embodiment of the tobacco sentiment analysis method of the present invention. Figure 1 As shown, the tobacco sentiment analysis method includes:

[0127] Step S1, building a tobacco sentiment analysis model;

[0128] Step S2, training the tobacco sentiment analysis model;

[0129] Step S3: input the pre-analyzed text into the trained tobacco sentiment analysis model to obtain the sentiment polarity of the text. The sentiment polarity includes positive, neutral and negative. For example, the output value of the tobacco sentiment analysis model is 0, indicating that the sentiment polarity is negative; the output value of the tobacco sentiment analysis model is 1, indicating that the sentiment polarity is neutral; and the output value of the tobacco sentiment analysis model is 2, indicating that the sentiment polarity is positive.

[0130] In one embodiment, Figure 2 As shown, step S1 includes:

[0131] Step S11: introduce the multi-head attention mechanism and use the pre-trained model Ernie to obtain the word vector matrix;

[0132] Step S12: Using the word vector matrix as input to a convolutional neural network (CNN), extracting a first eigenvector matrix, using the first eigenvector matrix as input to a bidirectional long short-term memory network (BiLSTM), obtaining semantic features and context information, and embedding the semantic features and context information into the first eigenvector matrix to obtain a second eigenvector matrix;

[0133] Step S13, obtaining the semantic-level attention and word-level attention of the second eigenvector matrix through a hierarchical attention network, and obtaining a third eigenvector matrix through a weighted combination of the second eigenvector matrix, the semantic-level attention, and the word-level attention;

[0134] Step S14: predict the sentiment polarity of the third eigenvector matrix using a softmax function.

[0135] The mixed vectors obtained by the CNN of the present invention are used as the input sequence of the LSTM, enriching the extraction of semantic features and making full use of contextual information. The LSTM model is processed bidirectionally to obtain BiLSTM, where the forward LSTM is used to model the above information and the backward LSTM is used to model the following information to obtain a context-related text representation. The present invention adopts a multi-layer attention network to distribute weights at the vocabulary and sentence levels. The attention network is used in both the output of the CNN and the BiLSTM to enhance the model's text feature extraction capabilities. The multi-layer attention network uses both semantic-level attention and word-level attention, simultaneously learning the importance of sentences and words, and obtaining the final weight distribution through corresponding aggregation operations. Inputting the output of the BiLSTM processing into the HAN layer can effectively improve the classification accuracy.

[0136] In one embodiment, step S11 includes:

[0137] Mark the sentences in the text to form a text sequence. The tags include [CLS] and [SEP]. [CLS] means it is placed at the beginning of a sentence, and [SEP] is used to separate two input sentences. Preferably, the text sequence is , n represents the length of the sentence, For the first words;

[0138] Embed the original words, fragments and sequence positions into the above text sequence;

[0139] Multi-head attention is introduced to obtain the correlation and importance between different words in different sentences in the embedded text sequence. Preferably, the correlation and importance between different words in different sentences are obtained by the following formula:

[0140]

[0141] Among them, Q (Query), K (Key), V (Value) is the input word vector matrix, is the input word vector dimension, For each word's relevance to other words, is the weighted sum of word vectors, is the importance of different words in different sentences, the larger the weighted sum, the more important it is; is the attention operation, and r is the head representation in the multi-head attention; that is, by Get the relationship between each word in the input sequence and other words by The vector is compressed and then normalized through the softmax function to obtain the weighted sum of the word vectors, which reflects the correlation and importance between different words in different sentence sequences.

[0142] Ernie outputs the latent variable matrix as the word vector matrix. Preferably, the latent variable matrix is , where h represents the latent variable dimension of Ernie.

[0143] In one embodiment, step S12 includes:

[0144] The word vector output by Ernie is used as the input of the convolutional neural network (CNN) to extract features and obtain the first eigenvector matrix. Preferably, convolution kernels of different sizes are used to extract features. More preferably, the convolution kernels of different sizes are 2, 3 and 4 respectively. Preferably, the nonlinear activation function ReLU is used between the layers of the convolutional neural network (CNN). Preferably, the first eigenvector matrix is obtained by the following formula:

[0145]

[0146] in, is the weight term, Represents the word vector matrix Go to The word vector of the row, is the bias term, is the first eigenvector matrix feature vectors;

[0147] Perform maximum pooling on the first eigenvector matrix. Preferably, perform maximum pooling on the first eigenvector matrix using the following formula:

[0148]

[0149] in, is the eigenvector of the first eigenvector matrix after pooling;

[0150] The first eigenvector matrix after maximum pooling is used as the input of a bidirectional long short-term memory network (BiLSTM) to obtain semantic features and context information, and the semantic features and context information are embedded into the first eigenvector matrix after maximum pooling to obtain a second eigenvector matrix.

[0151] In one embodiment, step S13 includes:

[0152] The third eigenvector matrix is obtained by the following formula:

[0153]

[0154]

[0155]

[0156] in, is the second eigenvector matrix, For the The vector representation of a sentence, is the bias term, is a randomly initialized vector, is the importance weight, The obtained weighted vector representation of the 𝑖th sentence is also the 𝑖th eigenvector of the third eigenvector matrix; yes The attribute representation of is the context vector, which can be randomly initialized and learned together during training, express The high-level representation obtained by weighted summation of the importance of is the vector representation of the entire text, which summarizes all the information of the sentences in the entire text.

[0157] In one embodiment, step S14 includes:

[0158] Use softmax to predict the sentiment polarity of the third eigenvector matrix by the following formula:

[0159]

[0160] Where M is the number of eigenvectors in the third eigenvector matrix. The Softmax function can be used to convert the output values of multiple classifications into a probability distribution between [0, 1] and 1, and the highest probability is selected as the output.

[0161] In one embodiment, step S2 further includes constructing a tobacco dataset before training the tobacco sentiment analysis model.

[0162] Figure 3 FIG. 1 is a flow chart of an embodiment of the method for constructing a tobacco dataset according to the present invention. Figure 3 As shown, the method for constructing a tobacco dataset includes:

[0163] Step S100, collecting multiple comment texts about tobacco;

[0164] Step S300: annotating the plurality of comment texts using different methods, including sentiment dictionaries and SnowNLP;

[0165] Step S400: Filter out annotations that are consistent with those in different annotation methods.

[0166] In one embodiment, before step S300, step S200 is further included to perform data preprocessing on the collected multiple comment texts, and the data preprocessing includes one or more of the following steps:

[0167] Remove duplicate comments, useless comments and / or advertisements;

[0168] Correct or delete typos;

[0169] Change traditional Chinese characters to simplified Chinese. For example, we collected posts on https: / / www.yanyue.cn. The comments on the forum are random, and most of them are in non-standard language with chaotic grammatical structure. The text contains typos, traditional Chinese characters, repeated comments, useless comments, advertisements, etc. After a series of data preprocessing, we obtained 75,610 valid online comments.

[0170] In one embodiment, step S400 includes:

[0171] Determine whether the annotations of multiple comment texts in different ways are consistent;

[0172] If the annotations are consistent, the annotations are considered reliable and added to the tobacco dataset.

[0173] If the annotations are inconsistent, cross-validation is performed on the inconsistent annotations.

[0174] Figure 4 FIG. 1 is a flow chart of an embodiment of the method for annotating multiple comment texts using a sentiment dictionary according to the present invention. Figure 4 As shown, the method of using a sentiment dictionary to annotate multiple comment texts includes:

[0175] Step S310, establishing a database, wherein the database includes a basic sentiment dictionary, a degree adverb dictionary, a conjunction dictionary, a negation word dictionary, a stop word dictionary, and a tobacco sentiment dictionary;

[0176] Step S320: Segment the review text into sentences and words to obtain each sentence in the review text, each clause in each sentence, and each word in each sentence and each clause. For example, the read text data is segmented into multiple clauses according to punctuation marks, and the clauses are segmented using Jieba.

[0177] Step S330: Analyze the modification relationship between the negative words and the sentiment words in the clause through the database, use the sentiment word dictionary, the degree adverb dictionary, and the negation word dictionary to identify the sentiment words (W1, W2, ..., Wn), degree adverbs (D1, D2, ..., Dn), negation words (I1, I2, ..., In), and transitional conjunctions (Z1, Z2, ..., Zn) in the clause, and record their positions. For each degree adverb and negation word, the nearest sentiment word in the direction is used as its modification target, and the modification relationship is determined according to the number of negation words.

[0178] Step S340, analyzing the tendency value of each sentence through the database;

[0179] Step S350: Accumulate the sentiment value of each sentence to obtain the positive sentiment tendency score and the negative sentiment tendency score of the comment text;

[0180] Step S360: In order to make the text sentiment tendency score able to represent the tendency degree of the text, the Sigmoid function is used to obtain the sentiment tendency, positive sentiment tendency degree and negative sentiment tendency degree of the comment text.

[0181] In one embodiment, the step of establishing a database in step S310 includes:

[0182] Use LDA topic model to select the topic of the review text;

[0183] Initialize the attribute word dictionary according to the above topics;

[0184] Use word2vec to obtain the distance between word vectors in the attribute word dictionary after initialization;

[0185] Classify the word vectors whose distance is greater than a set value into topics and construct a tobacco sentiment dictionary. Preferably, the set value is 0.8;

[0186] Build a basic sentiment dictionary, a degree adverb dictionary, a conjunction dictionary, a negation word dictionary, and a stop word dictionary.

[0187] Preferably, the HowNet sentiment dictionary is used as the basic sentiment dictionary; preferably, the Harbin Institute of Technology stop word dictionary is used as the stop word dictionary; preferably, the conjunction word list and the negation word list on the Internet are used to organize them into a conjunction dictionary and a negation word dictionary through data processing; preferably, a degree adverb dictionary is constructed based on the HowNet degree level words, and weights are assigned according to extreme, high, medium-high, medium and low quantities to obtain a degree adverb dictionary.

[0188] The present invention adopts the LDA topic model to perform topic modeling and output a dictionary. In order to be more in line with the sentiment analysis in the tobacco field, by customizing the topic and initializing the attribute word dictionary, the distance between word2vec word vectors is used to select attributes with a distance greater than 0.8 for topic classification. The two dictionaries are fused to obtain the tobacco field theme sentiment dictionary, and the basic sentiment dictionary, degree adverb dictionary, conjunction dictionary, negation word dictionary and stop word dictionary are combined for sentiment analysis.

[0189] In one embodiment, the step S330 of analyzing the modification relationship between the negative words and the sentiment words in the sentence through the database includes:

[0190] Using the tobacco sentiment dictionary, degree adverb dictionary, conjunction dictionary, and negation dictionary, we identify sentiment words, degree adverbs, negation words, and transitional conjunctions in the review texts and embed the positions of sentiment words, degree adverbs, negation words, and transitional conjunctions.

[0191] Get every adverb of degree and every negation;

[0192] The sentiment word closest to each degree adverb is used as the modification target of the degree adverb;

[0193] The sentiment word closest to each negative word is used as the modification target of the negative word;

[0194] The modification relationship is determined according to the number of negation words. When the number of times a negation word appears in a sentence is odd, the modification relationship of the negation word to the sentiment word is negative; when the number of times a negation word appears in a sentence is even, the modification relationship of the negation word to the sentiment word is positive.

[0195] In one embodiment, the step S340 of analyzing the tendency value of each sentence through the database includes:

[0196] The sentiment score of each clause is obtained by the following formula

[0197]

[0198] in, Indicates the modification relationship between negative words and sentiment words. Indicates the score of degree adverbs (there are score values in the degree adverb dictionary), 、 Represents the sentiment scores of sentiment words and clauses respectively (there are sentiment scores in the tobacco sentiment dictionary), Indicates the number of transitional conjunctions;

[0199] When a sentence includes multiple clauses, and both the clauses with positive sentiment scores and the clauses with negative sentiment scores are not 0, reduce the positive sentiment score and / or increase the negative sentiment score;

[0200] When the transitional conjunction is not 0, the sentiment score of the clause after the transitional conjunction is increased and / or the sentiment score of the clause before the transitional conjunction is decreased.

[0201] This method uses a database to analyze the tendency of each sentence to better align with the habits of online slang. When a sentence expresses both positive and negative sentiment, the negative tendency is usually greater than the positive. Therefore, when the positive and negative sentiment scores are both non-zero, the positive sentiment score is lowered. Transitional conjunctions indicate a more pronounced sentiment at the transition point, thus increasing the transition sentence score and decreasing the score before the transition.

[0202] In one embodiment, the step S360 of using the Sigmoid function to obtain the sentiment tendency, positive sentiment tendency degree, and negative sentiment tendency degree of the comment text includes:

[0203] The emotional tendency, positive emotional tendency and negative emotional tendency of the comment text are obtained by the following formula

[0204]

[0205]

[0206] in, Represents the positive sentiment score of the comment text, Indicates the negative sentiment score of the comment text, Indicates the percentage of negative sentiment scores, Indicates the percentage of positive sentiment score.

[0207] In one embodiment, step S2 of training the tobacco sentiment analysis model includes:

[0208] The sum of cross entropy loss and L2 regularization loss is used as the loss function. Preferably, the cross entropy loss and L2 regularization loss are obtained using the following formula:

[0209]

[0210]

[0211] in, is the cross entropy loss, is the third eigenvector matrix feature vectors, for Weighted, for Bias term, Ein is the error of the training sample without regularization, λ is the regularization parameter, which can be adjusted. is the weight parameter;

[0212] Input tobacco dataset into tobacco sentiment analysis model for training;

[0213] The tobacco sentiment analysis model corresponding to the minimum loss function is used as the trained tobacco sentiment analysis model.

[0214] In one embodiment, the step of constructing a tobacco dataset further includes:

[0215] The tobacco dataset is divided into training set, validation set and test set;

[0216] The tobacco sentiment analysis model is trained using the above training set;

[0217] Optimize validation set parameters, for example, by using grid optimization to optimize hyperparameters such as learning_rate and batch_size, and algorithm parameters such as convolution kernel size;

[0218] The tobacco sentiment analysis model after parameter tuning is evaluated using a test set, where the evaluation indicators include one or more of accuracy, precision, recall rate, and f1-score.

[0219] Figure 5 This is a schematic diagram of a structural block diagram of an embodiment of the tobacco sentiment analysis system of the present invention, as shown in FIG. Figure 5 As shown, the tobacco sentiment analysis system includes:

[0220] Model building part 1, building a tobacco sentiment analysis model;

[0221] Training unit 2, training the tobacco sentiment analysis model constructed by the above-mentioned model construction unit;

[0222] The prediction unit 3 inputs the pre-analyzed text into the tobacco sentiment analysis model trained by the training unit to obtain the sentiment polarity of the text, where the sentiment polarity includes positive, neutral, and negative;

[0223] Among them, the model construction part 1 includes an embedding layer 11, a feature extraction layer 12, a hierarchical attention network 13 and a sentiment polarity analysis model 14. The embedding layer introduces a multi-head attention mechanism and uses the pre-trained model Ernie to obtain a word vector matrix; the feature extraction layer includes a convolutional neural network (CNN) and a bidirectional long short-term memory network (BiLSTM). The word vector matrix of the embedding layer is the input of the convolutional neural network (CNN). The convolutional neural network extracts a first eigenvector matrix, which is the input of the bidirectional long short-term memory network (BiLSTM). The bidirectional long short-term memory network obtains semantic features and context information, and embeds the semantic features and context information into the first eigenvector matrix to obtain a second eigenvector matrix; the hierarchical attention network extracts the semantic-level attention and word-level attention of the second eigenvector matrix output by the feature extraction layer, and obtains a third eigenvector matrix by weighted combination of the second eigenvector matrix, the semantic-level attention and the word-level attention; the sentiment polarity analysis model predicts the sentiment polarity of the third eigenvector matrix output by the hierarchical attention network through a softmax function.

[0224] The embedding layer of the tobacco sentiment analysis system described in this paper uses the latest pre-trained model, Ernie, to obtain a word embedding matrix. A feature extraction layer based on CNN and RNN concepts is then employed to capture contextual information. Next, a hierarchical attention mechanism is introduced to focus on important words and sentences in layers and embed them into the input text. Finally, a sentiment classification module is used to predict the sentiment polarity of sentences.

[0225] In one embodiment, the embedding layer tokenizes the input sentence (e.g., using the Word Piece method to tokenize the input sentence). Special tokens [CLS] and [SEP] are added to the text sequence, thereby forming a text sequence of The final embedding of each subword is generated by combining its original word embedding, fragment embedding and sequence position embedding. The multi-head attention mechanism is introduced into the encoder structure of the Ernie model. After Ernie processing, the output is , the long ones are cut off and the short ones are lengthened.

[0226] In one embodiment, the feature extraction layer combines the concepts of CNN and RNN to more effectively capture and utilize contextual information. The output H from the Ernie encoder layer is sent to multiple CNNs, each of which includes convolution and pooling operations. In one embodiment, the CNN has three convolution kernel sizes: (2, 3, 4), with 100 convolution kernels for each size to extract features. For a convolution kernel with a convolution window size of d, :

[0227]

[0228] The activation function uses the nonlinear activation function ReLU, and the ReLU function is , Represents the segment of the input vector from position i to position i+d-1.

[0229] The pooling operation uses maximum pooling to extract the most obvious features.

[0230]

[0231] The mixed vectors generated by the CNN serve as the input sequence for the LSTM. To extract richer semantic features and fully utilize contextual information, the LSTM model is bidirectionally processed to create a BiLSTM. The forward LSTM is used to model the aforementioned information, and the backward LSTM is used to model the following information, resulting in a context-sensitive text representation.

[0232] In one embodiment, a hierarchical attention network distributes weights at both the vocabulary and sentence levels, and utilizes the attention network at the output of CNN and BiLSTM to enhance the model's text feature extraction capability.

[0233] The multi-layer attention network utilizes both semantic-level attention and word-level attention to simultaneously learn the importance of sentences and words, and obtains the final weight distribution through corresponding aggregation operations. Inputting the output of BiLSTM processing into HAN can effectively improve classification accuracy:

[0234] In one embodiment, the sentiment polarity analysis model consists of a linear layer and a softmax layer. Text feature information with different weights is fed into the softmax layer to predict sentiment polarity as the final output. The softmax layer normalizes the output probabilities and selects the unit with the highest probability as the output. Another dropout layer is performed before the softmax layer (forming a linear layer).

[0235] Due to the forum nature of tobacco, more people are willing to recommend good products and share experiences, followed by complaints about bad products. Through data processing and annotation, the present invention obtained 38,244 positive texts, 16,572 neutral texts, and 20,794 negative texts. Neutral sentences are particularly rare. Because data category imbalance is more consistent with the actual situation of the internet, data category balancing was not performed. The final tobacco dataset contained 75,610 data points, which were divided into training, validation, and test sets in an 8:1:1 ratio. The training set was used for model training, the validation set was used for parameter tuning, and the test set was used for evaluation. The statistical data of the tobacco dataset of the present invention are shown in Table 1.

[0236] Table 1

[0237]

[0238] To ensure the reliability of our results, we used the public dataset from the NLPCC2014 sentiment analysis task to verify its effectiveness. These reviews come from various fields, such as books, DVDs, and electronic products. The dataset is divided into training, validation, and test sets. Details of the dataset are shown in Table 2.

[0239] Table 2

[0240]

[0241] In all experiments, we used 300-dimensional word embeddings pre-trained from Sogou News (https: / / github.com / Embedding / Chinese-Word-Vectors) to initialize the word embeddings for non-BERT models. We trained these models using the Adam optimizer. We set the dropout probability for all layers to 0.1, the maximum sequence length to 128, the batch size to 64, and the learning rate to 5e. -5 The evaluation indicators include accuracy (ACC), precision (precision), recall (re-call), and the comprehensive index (f1-score) to reflect effectiveness.

[0242] Accuracy: ,

[0243] ,

[0244] ,

[0245] Composite Index: ,

[0246] Among them, TP: positive samples predicted by the model as positive; TN: negative samples predicted by the model as negative; FP: negative samples predicted by the model as positive; FN: positive samples predicted by the model as negative.

[0247] Experimental evaluations were conducted on both public datasets and tobacco-specific datasets, and compared with many current advanced research methods. The tobacco sentiment analysis method of the present invention achieved better and more outstanding results. The experimental results are shown in Table 3:

[0248] Table 3

[0249]

[0250] First, the experimental results show that the present invention consistently outperforms all baseline models in the prior art. On the tobacco dataset, the present invention improved the results of the second-best Ernie model by approximately 3.2%, and on the public nlpcc2014 task dataset, the present invention improved the results by approximately 2.6%. The present invention outputs three types of accuracy, recall, and f1-score indicator values: positive, neutral, and negative. It can be seen that the baseline models of the prior art are generally inaccurate in predicting neutral emotions. Due to data imbalance and the large gap, the accuracy of neutral sentences is lower than expected, which also proves that the accuracy increases with the increase in the amount of training data. The present invention can accurately predict positive and negative emotions, and at the same time improve the accuracy of neutral sentences by 7.54%.

[0251] The present invention also conducted ablation experiments. Specifically, the experiments further studied and demonstrated the impact of the various components of the present invention on model performance. The Ernie model, which does not have a feature extraction layer and a hierarchical attention mechanism, was used as the baseline model for the ablation experiment. Based on the baseline model, CNN, BiLSTM, and HAN network layers were added as variants to study the impact of different parts of the present invention. CNN was inserted before the classification part of the model to obtain the Ernie-CNN model. Similarly, BiLSTM was inserted before the classification part to obtain the Ernie-LSTM model. In addition, the present invention also used BERT to replace Ernie, and verified these ablation models on the dataset. The accuracy and F1-score ablation experiment results are shown in Table 4.

[0252] Table 4

[0253]

[0254] Table 4 shows that the model using Ernie as the encoder significantly outperforms the model using BERT as the encoder, demonstrating the effectiveness of the pre-trained Ernie model. Furthermore, Ernie-CNN-LSTM performs better than the baseline model, demonstrating the effectiveness of the CNN and LSTM combination. By adding a hierarchical attention mechanism, we obtain the EHCRNN model proposed in this paper, which achieves significant performance improvements on various datasets. This demonstrates not only the effectiveness of the hierarchical attention mechanism in aggregating sequence information, but also the effectiveness of the EHCRNN model's structural design.

[0255] The present invention also performs parameter estimation. Specifically, the present invention experiments the influence of different parameters on the model results. The parameter evaluation experiment is conducted on the nlpcc2014 task dataset. Figure 6 As can be seen, when the input size increases from 64 to 128, the accuracy improves significantly, but when it increases from 128 to 150, the improvement is less pronounced. Due to varying sentence lengths, the input padding size (the amount of padding) uses a short-to-long truncated approach, which may truncate important content within the sentence. While increasing the input padding size slightly improves accuracy, it also puts pressure on the server and significantly increases runtime. Therefore, the model achieves optimal performance when the input size is 128.

[0256] This paper combines the advantages of the pre-trained Ernie model with deep learning networks and introduces a hierarchical attention network to propose a sentiment analysis model, EHCRNN. Experimental results on a new tobacco dataset and the publicly available NLPCC2014 task dataset demonstrate that this tobacco sentiment analysis method achieves better classification results than currently studied algorithms and models.

[0257] The tobacco sentiment analysis method provided by the present invention is applied to an electronic device. In this embodiment, the electronic device can be a terminal with computing functions such as a server, a mobile phone, a tablet computer, a portable computer, or a desktop computer.

[0258] The electronic device includes a memory, a processor, a network interface and a communication bus.

[0259] The memory includes at least one type of readable storage medium. The at least one type of readable storage medium may be a non-volatile storage medium such as a flash memory, a hard disk, a multimedia card, a card-type memory, etc. In some embodiments, the readable storage medium may be an internal storage unit of the electronic device, such as a hard disk of the electronic device. In other embodiments, the readable storage medium may also be an external memory of the electronic device, such as a plug-in hard disk equipped on the electronic device, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc.

[0260] In this embodiment, the readable storage medium of the memory is generally used to store a tobacco sentiment analysis program installed in the electronic device, etc. The memory can also be used to temporarily store data that has been output or is to be output.

[0261] In some embodiments, the processor may be a central processing unit (CPU), a microprocessor, or other data processing chip, configured to execute program codes stored in a memory or process data, such as executing a tobacco sentiment analysis program.

[0262] The network interface may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface), which is generally used to establish a communication connection between the electronic device and other electronic user terminals.

[0263] The communication bus is used to realize the connection and communication between these components.

[0264] The above embodiment only shows one embodiment of the electronic device, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0265] Optionally, the electronic device may also include a user interface, which may include an input unit such as a keyboard, a voice input device such as a microphone, a user terminal with a voice recognition function, a voice output device such as a speaker, headphones, etc. Optionally, the user interface may also include a standard wired interface or a wireless interface.

[0266] Optionally, the electronic device may further include a display, which may also be referred to as a display screen or a display unit.

[0267] In some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an organic light-emitting diode (OLED) touch screen, etc. The display is used to display information processed in the electronic device and to display a visual user interface.

[0268] Optionally, the electronic device further includes a touch sensor. The area provided by the touch sensor for a user to perform a touch operation is referred to as a touch area. Furthermore, the touch sensor described herein may be a resistive touch sensor, a capacitive touch sensor, or the like. Furthermore, the touch sensor includes not only contact touch sensors but also proximity touch sensors, and the like. Furthermore, the touch sensor may be a single sensor or a plurality of sensors arranged, for example, in an array.

[0269] Optionally, the electronic device may further include logic gate circuits, sensors, audio circuits, etc., which will not be described in detail here.

[0270] The tobacco sentiment analysis method provided by the present invention is applied to a memory of a computer storage medium and may include a tobacco sentiment analysis program. When executed by a processor, the tobacco sentiment analysis method of the above-mentioned embodiments is implemented.

[0271] The tobacco sentiment analysis program 10 can also be divided into one or more modules, one or more modules are stored in a memory and executed by a processor to complete the present invention. The module referred to in the present invention refers to a series of computer program instruction segments that can perform a specific function.

[0272] The specific implementation of the computer-readable storage medium of the present invention is roughly the same as the specific implementation of the above-mentioned tobacco sentiment analysis method, system, and electronic device, and will not be repeated here.

[0273] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0274] The serial numbers of the embodiments of the present invention described above are for description only and do not represent the advantages and disadvantages of the embodiments. Through the description of the above implementation modes, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, including a number of instructions for enabling a terminal user terminal (which can be a mobile phone, computer, server, or network user terminal, etc.) to execute the methods described in each embodiment of the present invention.

[0275] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein with equivalents. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A tobacco sentiment analysis method, characterized in that: include: Step S1, building a tobacco sentiment analysis model; Step S2, training the tobacco sentiment analysis model; Step S3, inputting the pre-analyzed text into the trained tobacco sentiment analysis model to obtain the sentiment polarity of the text, where the sentiment polarity includes positive, neutral, and negative; Wherein, the step S1 includes: Step S11: introduce the multi-head attention mechanism and use the pre-trained model Ernie to obtain the word vector matrix; Step S12: Using the word vector matrix as input to the convolutional neural network, extracting a first eigenvector matrix, using the first eigenvector matrix as input to the bidirectional long short-term memory network, obtaining semantic features and context information, and embedding the semantic features and context information into the first eigenvector matrix to obtain a second eigenvector matrix; Step S13, obtaining the semantic-level attention and word-level attention of the second eigenvector matrix through a hierarchical attention network, and obtaining a third eigenvector matrix through a weighted combination of the second eigenvector matrix, the semantic-level attention, and the word-level attention; Step S14: predict the sentiment polarity of the third eigenvector matrix using a softmax function.

2. The tobacco sentiment analysis method according to claim 1, characterized in that The step S11 includes: Mark the sentences in the text to form a text sequence. The tags include [CLS] and [SEP]. [CLS] means it is placed at the beginning of a sentence, and [SEP] is used to separate two input sentences. The text sequence is , n represents the length of the sentence, For the first words; Embed the original words, fragments and sequence positions into the above text sequence; Introducing multi-head attention to obtain the correlation and importance between different words in different sentences in the embedded text sequence; The latent variable matrix output by Ernie is used as the word vector matrix. The latent variable matrix is , where h represents the latent variable dimension of Ernie.

3. The tobacco sentiment analysis method according to claim 2, characterized in that The correlation and importance between different words in different sentences are obtained by the following formula: Among them, Q, K, V are input word vector matrices, is the input word vector dimension, For each word's relevance to other words, is the weighted sum of word vectors, is the importance of different words in different sentences, the larger the weighted sum, the more important it is; is the attention operation, and r is the head representation in the multi-head attention.

4. The tobacco sentiment analysis method according to claim 1, characterized in that The step S12 includes: The word vector output by Ernie is used as the input of the convolutional neural network to extract features and obtain the first eigenvector matrix; Perform maximum pooling on the first eigenvector matrix.

5. The tobacco sentiment analysis method according to claim 1, characterized in that The step S13 includes: The third eigenvector matrix is obtained by the following formula: in, is the second eigenvector matrix, For the The vector representation of a sentence, is the bias term, is a randomly initialized vector, is the importance weight, The obtained The weighted vector representation of the sentence based on weights is also the third eigenvector matrix feature vectors.

6. The tobacco sentiment analysis method according to claim 1, characterized in that: The step S14 includes: Use softmax to predict the sentiment polarity of the third eigenvector matrix by the following formula: Wherein, M is the number of eigenvectors of the third eigenvector matrix.

7. The tobacco sentiment analysis method according to claim 4, characterized in that: In the step of using the word vector output by Ernie as the input of the convolutional neural network, extracting features, and obtaining the first eigenvector matrix: convolution kernels of different sizes are used to extract features.

8. The tobacco sentiment analysis method according to claim 7, characterized in that: The convolution kernels of different sizes are 2, 3 and 4 respectively.

9. The tobacco sentiment analysis method according to claim 4, characterized in that: The nonlinear activation function ReLU is used between the layers of the convolutional neural network.

10. The tobacco sentiment analysis method according to claim 9, characterized in that: The first eigenvector matrix is obtained by the following formula: in, is the weight term, Represents the word vector matrix Go to The word vector of the row, is the bias term, is the first eigenvector matrix feature vectors.

11. The tobacco sentiment analysis method according to claim 4, characterized in that: The first eigenvector matrix is max-pooled using the following formula: in, is the eigenvector of the first eigenvector matrix after pooling; The first eigenvector matrix after maximum pooling is used as the input of a bidirectional long short-term memory network to obtain semantic features and context information, and the semantic features and context information are embedded into the first eigenvector matrix after maximum pooling to obtain a second eigenvector matrix.

12. The tobacco sentiment analysis method according to claim 1, characterized in that: Before step S2, the step also includes constructing a tobacco dataset.

13. The tobacco sentiment analysis method according to claim 12, characterized in that: The steps of constructing the tobacco dataset include: Collect multiple comment texts about tobacco; Annotating the plurality of review texts using different methods, including sentiment dictionaries and SnowNLP; Filter out annotations that are consistent in different ways.

14. The tobacco sentiment analysis method according to claim 13, characterized in that: The step of constructing the tobacco dataset further includes: Perform cross-validation on inconsistently labeled annotations.

15. The tobacco sentiment analysis method according to claim 14, characterized in that: The step S2 comprises: The sum of cross entropy loss and L2 regularization loss is used as the loss function; Input tobacco dataset into tobacco sentiment analysis model for training; The tobacco sentiment analysis model corresponding to the minimum loss function is used as the trained tobacco sentiment analysis model.

16. The tobacco sentiment analysis method according to claim 12, characterized in that: Also includes: The tobacco dataset is divided into training set, validation set and test set; The tobacco sentiment analysis model is trained using the above training set; Parameter tuning through validation set; The tobacco sentiment analysis model after parameter tuning is evaluated on a test set, where the evaluation indicators include one or more of accuracy, precision, recall rate, and f1-score.

17. The tobacco sentiment analysis method according to claim 13, characterized in that: The steps of marking the above-mentioned multiple comment texts in different ways include: Establishing a database, wherein the database includes a basic sentiment dictionary, a degree adverb dictionary, a conjunction dictionary, a negation word dictionary, a stop word dictionary and a tobacco sentiment dictionary; Segment the review text into sentences and words to obtain each sentence in the review text, each clause in each sentence, and each word in each sentence and each clause; Analyze the modification relationship between negative words and sentiment words in sentences through the database; Analyze the tendency value of each sentence through the database; Accumulate the sentiment value of each sentence to obtain the positive sentiment tendency score and negative sentiment tendency score of the comment text; The Sigmoid function is used to obtain the sentiment tendency, positive sentiment tendency degree and negative sentiment tendency degree of the review text.

18. The tobacco sentiment analysis method according to claim 17, characterized in that: In the step of segmenting the comment text into sentences and words, jieba is used to segment the clauses into words.

19. The tobacco sentiment analysis method according to claim 17, characterized in that: The steps of establishing a database include: Take the topic of the review text through the LDA topic model; Initialize the attribute word dictionary according to the above topics; Use word2vec to obtain the distance between word vectors in the attribute word dictionary after initialization; The word vectors whose distance is greater than the set value are classified into topics and a tobacco sentiment dictionary is constructed; Build a basic sentiment dictionary, a degree adverb dictionary, a conjunction dictionary, a negation word dictionary, and a stop word dictionary.

20. The tobacco sentiment analysis method according to claim 17, characterized in that: The step of analyzing the modification relationship between the negative words and the sentiment words in the sentence through the database includes: Using the tobacco sentiment dictionary, degree adverb dictionary, conjunction dictionary, and negation dictionary, we identify sentiment words, degree adverbs, negation words, and transitional conjunctions in the review texts and embed the positions of sentiment words, degree adverbs, negation words, and transitional conjunctions. Get every adverb of degree and every negation; The sentiment word closest to each degree adverb is used as the modification target of the degree adverb; The sentiment word closest to each negative word is used as the modification target of the negative word; The modification relationship is determined according to the number of negation words. When the number of times a negation word appears in a sentence is odd, the modification relationship of the negation word to the sentiment word is negative; when the number of times a negation word appears in a sentence is even, the modification relationship of the negation word to the sentiment word is positive.

21. The tobacco sentiment analysis method according to claim 17, characterized in that: The step of analyzing the tendency value of each sentence through the database includes: The sentiment score of each clause is obtained by the following formula in, Indicates the modification relationship between negative words and sentiment words. The score of the adverb of degree, 、 Represent the sentiment scores of sentiment words and clauses respectively, Indicates the number of transitional conjunctions; When a sentence includes multiple clauses and both the clauses with positive sentiment scores and the clauses with negative sentiment scores are not 0, reduce the positive sentiment score and / or increase the negative sentiment score; When the transitional conjunction is not 0, the sentiment score of the clause after the transitional conjunction is increased and / or the sentiment score of the clause before the transitional conjunction is decreased.

22. The tobacco sentiment analysis method according to claim 17, characterized in that: The step of using the Sigmoid function to obtain the sentiment tendency, positive sentiment tendency degree, and negative sentiment tendency degree of the comment text includes: The emotional tendency, positive emotional tendency and negative emotional tendency of the comment text are obtained by the following formula in, Represents the positive sentiment score of the comment text, Indicates the negative sentiment score of the comment text, Indicates the percentage of negative sentiment scores, Indicates the percentage of positive sentiment score.

23. The tobacco sentiment analysis method according to claim 19, characterized in that: The set value in the step of subject-classifying word vectors whose distance is greater than the set value is 0.

8.

24. A tobacco sentiment analysis system, characterized in that: include: Model building department, building tobacco sentiment analysis model; The training department trains the tobacco sentiment analysis model built by the model building department; The prediction unit inputs the pre-analyzed text into the tobacco sentiment analysis model trained by the training unit to obtain the sentiment polarity of the text, where the sentiment polarity includes positive, neutral, and negative; Among them, the model construction part includes an embedding layer, a feature extraction layer, a hierarchical attention network and a sentiment polarity analysis model. The embedding layer introduces a multi-head attention mechanism and uses the pre-trained model Ernie to obtain a word vector matrix; the feature extraction layer includes a convolutional neural network and a bidirectional long short-term memory network. The word vector matrix of the embedding layer is the input of the convolutional neural network. The convolutional neural network extracts a first eigenvector matrix, which is the input of the bidirectional long short-term memory network. The bidirectional long short-term memory network obtains semantic features and context information, and embeds the semantic features and context information into the first eigenvector matrix to obtain a second eigenvector matrix; the hierarchical attention network extracts the semantic-level attention and word-level attention of the second eigenvector matrix output by the feature extraction layer, and obtains a third eigenvector matrix by weighted combination of the second eigenvector matrix, the semantic-level attention and the word-level attention; the sentiment polarity analysis model predicts the sentiment polarity of the third eigenvector matrix output by the hierarchical attention network through a softmax function.

25. An electronic device, characterized in that: It includes a memory and a processor, wherein a tobacco sentiment analysis program is stored in the memory, and when the tobacco sentiment analysis program is executed by the processor, the steps of the tobacco sentiment analysis method according to any one of claims 1 to 23 are implemented.

26. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a tobacco sentiment analysis program. When the tobacco sentiment analysis program is executed by a processor, the steps of the tobacco sentiment analysis method according to any one of claims 1 to 23 are implemented.

Images