Semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis
Through the model of combining BiLSTM and multi-channel bidirectional graph convolutional network connection, the problem of insufficient utilization and overfitting of grammatical information in existing aspect-level sentiment analysis is solved, and the accuracy of emotional polarity judgment is improved. performance.
Patent Information
- Application Number
- CN202311786028.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-22
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-12-22
AI Technical Summary
The existing aspect-level sentiment analysis model ignores the multiple relationship characteristics between words and syntactic dependency tree information, resulting in inaccurate judgment of emotional polarity and the overfitting problem exists.
A composite model combining a random dense residual connection BiLSTM and a multi-channel bidirectional graph convolution network is adopted. Syntax features are extracted through syntax dependence trees, multiple relationship features are considered, and the syntax information of the intermediate BiGCN layer is used to enhance semantic features. express.
It effectively solves the problems of insufficient utilization and overfitting of grammatical information, improves the accuracy and overall performance of aspect-level sentiment analysis, and can better capture semantic and relational characteristics in sentences.
Smart Images

Figure CN117764082B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of semantic sentiment analysis, and in particular to a semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis. Background Art
[0002] Aspect-level sentiment analysis aims to predict the sentiment polarity of aspect words in a sentence. In recent years, graph convolutional network models have achieved very satisfactory results in the field of aspect-level sentiment analysis. However, previous studies only focused on low-level semantic features and ignored the existence of multiple relationship features between words, which are the key factors restricting the effectiveness of aspect-level sentiment analysis tasks.
[0003] Sentiment analysis is one of the basic tasks in the field of natural language processing. Sentiment analysis, also known as sentiment classification or opinion mining, is to obtain people's opinions, attitudes, and emotions by classifying texts. Sentiment analysis is a classification technology with great practical value and is widely used in some real-world scenarios, such as product recommendations and movie box office predictions.
[0004] Sentiment analysis can be divided into coarse-grained and fine-grained sentiment analysis according to its granularity. Coarse-grained sentiment analysis can find out the sentiment polarity of documents, paragraphs or sentences. Fine-grained sentiment analysis is also called aspect-level sentiment analysis. Aspect-level sentiment analysis targets one or more aspect words in a sentence and finds out the sentiment polarity of each aspect word. Aspect words usually refer to entities or entity attributes. Sentiment polarity includes positive, negative and neutral. Aspect-level sentiment analysis can judge the sentiment polarity of sentences in specific aspects more carefully and make the judgment of the sentiment polarity of texts more accurate. This makes aspect-level sentiment analysis a key research direction in the current field of sentiment analysis.
[0005] At present, neural network models are the mainstream method for aspect-level sentiment analysis. Convolutional neural networks are used for aspect-level sentiment analysis. However, CNN-based models can only extract features from continuous words, which is not enough to determine the sentiment polarity of aspect words described by multiple non-adjacent words. This may cause distant sentiment words related to aspect words to be ignored. Various attention-based mechanisms combined with LSTM are used to achieve this goal and achieve good results. However, it ignores the grammatical relationship between aspect words and context. In order to solve this problem, a dependency tree is added to the attention mechanism, and some syntactic constraints are added to the attention weights. However, the role of syntactic information has not been fully utilized. It is first proposed Using graph convolutional networks for aspect-level sentiment analysis and extracting grammatical features in the syntactic dependency tree can largely make up for the shortcomings of insufficient grammatical feature extraction by CNN and attention mechanisms. GCN can handle the situation where non-continuous words are used to describe the sentiment polarity of aspects. GCN on the dependency tree will collect non-continuous words into a smaller range and appropriately aggregate their features through graph convolution. However, previous GCN-based models only focused on low-level semantic features and ignored the existence of multiple relationship features between words. In addition, only the output of the last layer of GCN is used for aspect-level sentiment analysis. Some important grammatical information encoded by other GCN layers is ignored, and the syntactic dependency tree information is not fully utilized.
[0006] Therefore, a composite model combining random dense residual connection BiLSTM and multi-channel bidirectional graph convolutional network is proposed. Summary of the invention
[0007] The purpose of the invention provided by the present invention is to provide a semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis to solve the problems in the above-mentioned background technology.
[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions: a semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis, comprising:
[0009] Model training: By learning and training the data, the system learns key features and rules from the data and applies them to new training data;
[0010] Sentiment analysis: analyzing the probability of words in various sentiment polarities;
[0011] Attention mechanism: extract important sentiment words related to the sentiment polarity of aspect words from the sentence context, and set attention weights for each context word;
[0012] Random dense residual connection BiLSTM: mining temporal information and semantic information in data;
[0013] Word embedding: converting words in a sentence into vectorized representations;
[0014] Mask layer: mask the non-aspect words in the sentence and only retain the vector representation of the aspect words;
[0015] Multi-channel bidirectional graph convolutional network: used to better capture the features in the input data;
[0016] The random dense residual connection BiLSTM includes a stacked BiLSTM and a random dense residual connection, and a bidirectional signal connection is formed between the stacked BiLSTM and the random dense residual connection. The multi-channel bidirectional graph convolution network includes a syntactic dependency tree, a graph convolution network, a multi-channel graph convolution network, a bidirectional graph convolution network, and a residual connection. The syntactic dependency tree, the graph convolution network, the multi-channel graph convolution network, the bidirectional graph convolution network, and the residual connection are all bidirectionally signal connected.
[0017] The output end of the random dense residual connection in the random dense residual connection BiLSTM is bidirectionally connected to the input end of the model training, and the output end of the residual connection in the multi-channel bidirectional graph convolutional network is bidirectionally connected to the input end of the model training.
[0018] Stacked BiLSTM:
[0019]
[0020] c represents the number of BiLSTM layers. After encoding the word vector using stacked BiLSTM, the sentence context representation H is obtained. c for:
[0021]
[0022] The representation of each layer is recursively passed to all subsequent layers, and then the input of the next layer is represented by linearly superimposing the hidden state of the subsequent layer and the hidden states of all its previous layers, and the dense residual connections are randomly dropped during training.
[0023] The original dense residual connection of the stacked BiLSTM has the following form:
[0024]
[0025] Each layer of dense residual connection adds a random parameter r c , r of each layer c Deciding to remove or retain dense residual connections and adding randomization parameters is as follows:
[0026]
[0027] Random parameter r cThe value can only be 0 or 1. c = 0, the corresponding dense residual connection is deleted. c = 1, the corresponding dense residual connection is retained, r c is generated by a Bernoulli distribution.
[0028] The higher the BiLSTM layer, the more important the extracted semantic features are, and the greater the weight they occupy in the final output. Therefore, a weight parameter W is set for each layer. c ,The higher the BiLSTM layer, the larger the corresponding weight parameter, W c The representation is as follows:
[0029] c represents the cth layer, C is the total number of layers, q is the global parameter, and the weight coefficient W is added c After that, it looks like this:
[0030]
[0031] Furthermore, the output signal of the word embedding is connected to the input of the random dense residual connection in the random dense residual connection BiLSTM.
[0032] Furthermore, the output end of the word embedding is bidirectionally signal-connected to the input end of the model training.
[0033] Furthermore, the output signal of the random dense residual connection in the random dense residual connection BiLSTM is connected to the input of the attention mechanism.
[0034] Furthermore, the output signal of the random dense residual connection in the random dense residual connection BiLSTM is connected to the input of the mask layer.
[0035] Furthermore, the output end of the attention mechanism is bidirectionally signal-connected to the input end of the model training.
[0036] Furthermore, the output signal of the attention mechanism is connected to the input signal of the sentiment analysis.
[0037] Furthermore, the output signal of the residual connection in the multi-channel bidirectional graph convolutional network is connected to the input of the mask layer.
[0038] Furthermore, the output end of the mask layer is bidirectionally signal-connected to the input end of the model training.
[0039] Furthermore, the output end of the mask layer is bidirectionally signal-connected to the input end of the attention mechanism.
[0040] The present invention provides a disease network consultation system and a method for using the same. It has the following beneficial effects:
[0041] This semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis can extract deeper semantic features from shallower semantic features through random dense residual connection BiLSTM, enhance semantic feature representation, and effectively solve the overfitting problem of stacked BiLSTM, and take into account multiple relationship features between words.
[0042] This semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis can take into account the different relationship features between words through a multi-channel bidirectional graph convolutional network, encode and fuse different relationship features, thereby enhancing the overall performance of the model in aspect-level sentiment analysis.
[0043] This semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis can make full use of the useful grammatical information encoded in the intermediate BiGCN layer through residual connections to address the problem of inefficient use of grammatical information in the GCN-based model, and effectively enhance BiGCN's use of syntactic dependency tree information. This helps to make the grammatical connection between aspect words and context closer, and solves the problem of inefficient use of syntactic dependency tree information in GCN.
[0044] This semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis can interact aspect word vectors and context vectors through the attention mechanism, retrieve the sentiment features related to each aspect word in the context representation, and generate the final aspect word vector representation. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 It is an overall system diagram of the semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis of the present invention;
[0046] Figure 2 A schematic diagram of a random dense residual connection BiLSTM of a semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis of the present invention;
[0047] Figure 3 A schematic diagram of a multi-channel bidirectional graph convolutional network of a semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis of the present invention;
[0048] Figure 4 It is a schematic diagram of a model framework of a semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis of the present invention;
[0049] Figure 5 A schematic diagram of a stacked BiLSTM framework of a semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis of the present invention;
[0050] Figure 6 A schematic diagram of a framework of a random dense residual connection BiLSTM of a semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis of the present invention;
[0051] Figure 7 A schematic diagram of a syntactic dependency tree structure of a semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis according to the present invention;
[0052] Figure 8 A schematic diagram of a sentence analysis using a syntactic dependency tree of a semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis according to the present invention;
[0053] Fig. 9 Schematic diagram of three language relationship features of the semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis of the present invention;
[0054] Fig.10 A schematic diagram of a top-down and bottom-up syntactic dependency tree structure of a semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis of the present invention;
[0055] Fig.11 Schematic diagram of a bidirectional residual graph convolutional neural network of a semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis of the present invention.
[0056] In the figure: 1. Model training; 2. Sentiment analysis; 3. Attention mechanism; 4. Random dense residual connection BiLSTM; 5. Word embedding; 6. Mask layer; 7. Multi-channel bidirectional graph convolutional network. DETAILED DESCRIPTION
[0057] The technical scheme in the embodiments of the present invention will be clearly described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.
[0058] Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0059] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:
[0060] See also Figure 1-11 , the present invention provides a technical solution: a semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis:
[0061] The mainstream methods of aspect-level sentiment analysis can be divided into two categories: traditional machine learning and deep learning. Traditional machine learning methods include naive Bayes, support vector machine, maximum entropy and k-nearest neighbor, etc. Among them, support vector machine performs best in aspect-level sentiment analysis. However, traditional machine learning methods need to do a lot of feature engineering work when extracting the sentiment features of aspect words and their contexts, which consumes a lot of manpower and time. Compared with traditional machine learning methods, the deep neural network model has a more complex structure. It does not require manual feature engineering and can automatically capture important sentiment features in the text. Therefore, the deep neural network model has become the most mainstream method in the field of aspect-level sentiment analysis.
[0062] At present, most studies build neural network models on the whole sentence, trying to explore the relationship between aspect words and their context words. In terms of CNN, CNN is used for the first time in aspect-level sentiment analysis. A parameterized convolutional neural network is proposed, which uses filters and parameterized gates to add aspect information to the convolutional neural network. A model combining gated units and convolutional layers is proposed. The two convolutional layers process aspect information and sentiment information in parallel respectively. The gated unit selectively outputs sentiment features based on the given aspect words. However, the CNN model can only extract the features of adjacent words through convolution, and cannot determine the sentiment represented by non-adjacent words, and cannot consider grammatical information. In terms of RNN, G RNN is used to model the aspect-level sentiment analysis and has achieved good performance. Although RNN can process long text data, it has the problems of gradient disappearance and gradient explosion. Using LSTM or BiLSTM can effectively solve the shortcomings of RNN, but it still does not take grammatical information into consideration. People have realized that the combination of grammatical dependency tree and LSTM can take grammatical information into consideration and achieve better results. LSTM with grammatical dependency tree performs better than CNN. The attention mechanism is combined with LSTM to capture the sentiment features related to aspect words. Later, an interactive attention network was proposed to interactively learn context and aspect words, and generate representations of context and aspect words respectively.
[0063] In recent years, graph convolutional neural networks have attracted more and more attention in the field of artificial intelligence and have been applied to aspect-level sentiment analysis. It is very effective to encode syntactic information by combining graph convolutional neural networks with grammatical dependency trees. Marcheggiani claimed that GCN can be regarded as a supplement to LSTM, showing how GCN can be used to integrate syntactic information in neural models, and proposed a semantic role labeling model based on GCN. Graph convolution of grammatical dependency trees was used in relation extraction. GCN was used for aspect-level sentiment analysis and achieved good results. However, GCN has the problems of over-smoothing and insufficient extraction of semantic features. The syntactic dependency tree of the sentence is utilized by constructing a bidirectional graph convolutional network on the sentence, and an edge enhancement module is added to eliminate the noise in the syntactic dependency tree. However, these methods generally do not take into account the different relational features between words and the insufficient use of syntactic dependency tree information. In addition, previous studies often have simple representations of contextual semantic features and shallow representation levels of semantic features, which limits the further improvement of overall performance.
[0064] Task Definition
[0065] For a sentence containing n words and m aspect words
[0066] S={w1,w2,…,w τ+1 ,…,w t+m ,…,w n-1 ,w n},The goal of aspect-level sentiment analysis is to infer the sentiment polarity of m aspect words in sentence S, and the sentiment polarity can be positive, negative or neutral.
[0067] Word Embedding
[0068] Word embedding is the process of converting words in a sentence into vectorized representations, which is called word vector representation. The word vectors output by word embedding will be used as input to the subsequent neural network layer. The word embedding technology used is GloVe, which is an unsupervised learning algorithm for distributed representation of input words.
[0069] Given a sentence
[0070] S={w1,w2,…,w τ+1 ,…,w t+m ,…,w n-1 ,w n}, input each word of the sentence into the embedding lookup matrix generated by GloVe Search in to get the word vector representation of each word in the sentence The word vector representation of the entire sentence is X = {X1,X2,…X τ+1 ,…,X τ+m,…,X n-1 ,X n}, where |V| is the size of the vocabulary, d e represents the dimension of word embedding, and m represents the number of aspect words.
[0071] Random Dense Residual Connection BiLSTM
[0072] Stacked BiLSTM
[0073] After the sentence is processed by word embedding, it obtains the initial word vector representation. Some previous studies will process the initial word vector representation of the sentence through GRU or LSTM to obtain the shallow semantic feature representation of the sentence, and the obtained semantic features are limited. In order to overcome the shortcomings of previous studies that can only obtain shallow semantic feature representation, stacked BiLSTM is used to replace the previous model. The sentence is first processed by GloVe to obtain the initial word vector representation, and the initial word vector representation is then processed by the first layer of BiLSTM to obtain the shallow semantic feature representation of the sentence context. Then the shallow semantic feature representation is processed by the second layer of BiLSTM to obtain a higher-level semantic feature representation of the context, and so on. Specifically, the sentence processed by the word embedding layer is first used as the input of the first layer of BiLSTM. The first layer of BiLSTM generates a hidden state vector, which is used as the input of the second layer of BiLSTM. The representation method of the hidden state vector is as follows:
[0074] Single-layer BiLSTM:
[0075]
[0076] x t represents the initial input of BiLSTM, that is, the word vector representation generated by word embedding, represents the hidden state vector at time step t from the start of the BiLSTM, where d h is the dimension of the hidden state vector output by LSTM, 2×d h is the dimension of the hidden state vector output by BiLSTM. A single-layer BiLSTM consists of a forward LSTM and a backward LSTM. The forward LSTM inputs the sentence from left to right into the LSTM for encoding, while the backward LSTM inputs the sentence from right to left and finally outputs the hidden state vector in the opposite direction. and Combine to get h t ,h t Contains complete semantic information in both forward and backward directions.
[0077] Stacked BiLSTM:
[0078]
[0079] c represents the number of BiLSTM layers. After encoding the word vector using stacked BiLSTM, the sentence context representation H is obtained. c for:
[0080]
[0081] Random Dense Residual Connections
[0082] As the number of stacked BiLSTM layers increases, although deep semantic features of the context can be extracted, it also brings about the overfitting problem. Inspired by previous work, a novel random dense residual connection is designed for the stacked BiLSTM. The key idea is to recursively pass the representation of each layer to all subsequent layers, and then represent the input of the next layer by linearly superimposing the hidden state of the subsequent layer and the hidden state of all its previous layers, and randomly discard the dense residual connection during the training process.
[0083] The original dense residual connection of the stacked BiLSTM has the following form:
[0084]
[0085] In order to improve the generalization of the model and solve the overfitting problem, each layer of dense residual
[0086] Connect and add random parameter r c , r of each layer c Deciding to remove or retain dense residual connections and adding randomization parameters is as follows:
[0087]
[0088] Random parameter r c Only takes 0 or 1. c = 0, the corresponding dense residual connection is deleted. c = 1, the corresponding dense residual connection is retained, r c is generated by a Bernoulli distribution.
[0089] The higher the BiLSTM layer, the more important the extracted semantic features are, and the greater the weight they occupy in the final output. Therefore, a weight parameter W is set for each layer. c ,The higher the BiLSTM layer, the larger the corresponding weight parameter, W c The representation is as follows:
[0090] c represents the cth layer, C is the total number of layers, q is the global parameter, and the weight coefficient W is added c After that, it looks like this:
[0091]
[0092] Multi-channel Bidirectional Graph Convolutional Network
[0093] Syntactic dependency tree
[0094] The syntactic dependency tree is used to analyze the grammatical structure of a sentence and represent it as an easy-to-understand structure. The syntactic dependency tree constrains the syntactic dependency relationship between words in a sentence into a tree structure. The syntactic dependency tree converts the sentence into a directed graph. Each word in the sentence serves as a node of the directed graph. A sentence containing n words has n nodes.
[0095] Graph Convolutional Networks
[0096] The grammatical dependency tree converts sentences into directed graphs. Directed graphs are non-Euclidean data. CNN can only process Euclidean data, but not non-Euclidean data. Graph convolutional networks can be used to process non-Euclidean data. Specifically, GCN is used on the syntactic dependency tree. The syntactic dependency tree can provide grammatical constraints for aspect words and reduce the distance between aspect words and important sentiment words in the context. Then GCN is used to extract grammatical features. For a directed graph containing n nodes, its adjacency matrix A∈R can be obtained. n×n , which represents the syntactic dependency information between words. The nodes of GCN are represented as follows:
[0097] represents the word vector representation of the i-th node of the l-th layer GCN, δ is the activation function, the ReLu function is used here, A ij Represents node h i With h j Whether they are adjacent or not also indicates whether there is a grammatical dependency relationship between them. ij =1, indicating that node h i With h j There is a syntactic dependency relationship, when A ij =0, indicating h i With h j There is no syntactic dependency, where W l is the linear transformation weight, b l It is biased.
[0098] Multi-channel graph convolutional network
[0099] Considering that there are many different relational features between words in a sentence, a multi-channel graph convolutional network is used to process various relational features between words. One relationship corresponds to one channel. Three types of relational features are introduced for each word, including syntactic dependency type, relative aspect distance and random edge elimination. For the syntactic dependency type, if the word h i With hj When there is a syntactic relationship, A ij = 1, for relative aspect distance, a relative weight coefficient W is added to each word i , the closer the distance is to the aspect word, the more important the sentiment polarity judgment is. Therefore, the closer the distance is to the aspect word, the larger the relative weight coefficient is set, and the relative weight coefficient of the aspect word is set to 0, where F(·) is the function of assigning position weights, and the F(·) function is expressed as follows:
[0100]
[0101] For random edge elimination, random parameters are added to the edges in the sentence directed graph. The random parameters take 1 or 0. 1 means the edge is retained, and 0 means the edge is eliminated. By randomly eliminating edges, overfitting and oversmoothing can be prevented. The random parameters follow the Bernoulli distribution. Finally, the nodes of the three channels are represented and Linear superposition is performed in the form of:
[0102] represents the node representation processed by the syntactic dependency tree, represents the node representation after relative aspect distance processing, Represents the node representation after random edge elimination.
[0103] Bidirectional Graph Convolutional Network
[0104] In order to solve the problem that GCN ignores the direction of grammatical information propagation, a bidirectional graph convolutional network is used to replace GCN. Compared with GCN, BiGCN considers both the top-down propagation information and the bottom-up aggregation information on the syntactic dependency tree, which can extract the grammatical information of the context more comprehensively. Each layer of BiGCN contains two layers of GCN, one layer of GCN is applied to the top-down syntactic dependency tree, and the other layer is applied to the bottom-up syntactic dependency tree, and finally their outputs are merged.
[0105] In this section, BiGCN is applied to syntactic dependency trees with different sentence propagation directions to extract grammatical information of the context. BiGCN updates the directed graph nodes as follows:
[0106]
[0107] and They represent the node representations after BiGCN extracts grammatical information from top to bottom and bottom to top on the dependency tree, A is the adjacency matrix, and A T is the transposed matrix of A.
[0108] Iterate from two directions respectively, and finally splice the nodes in the two directions and Get the node representation of BiGCN It is expressed as follows:
[0109] Residual Connection
[0110] As the number of CNN and RNN network layers increases, the gradient vanishing and gradient exploding problems will occur, and the model is prone to overfitting. Previous studies have shown that introducing residual connections in CNN and RNN can solve these problems well. Inspired by this, residual connections are designed in multi-channel bidirectional graph convolutional networks to solve the defect of insufficient extraction of grammatical information.
[0111] Previous GCN-based models only focused on the output of the last layer of GCN for aspect-level sentiment analysis. Some important grammatical information encoded in other GCN layers was ignored, resulting in grammatical information loss and limited utilization of grammatical information. Introducing residual connections into BiGCN and fusing the grammatical features of low-layer BiGCN with those of high-layer BiGCN can solve the problem of insufficient utilization of syntactic information. In this study, the input and output of the BiGCN in the previous layer are linearly superimposed as the input of the next layer of BiGCN. The BiGCN expression with residual connections is:
[0112]
[0113] The BiGCN layer output is
[0114] Mask layer
[0115] The output of BiGCN is used as the input of the mask layer to mask the non-aspect words in the sentence and only retain the aspect word vector representation:
[0116] The output of the mask layer is as follows: It is the word vector representation of the aspect word, which contains the grammatical relationship between the aspect word and the context, and prepares for the subsequent attention mechanism to extract the sentiment features of the aspect word.
[0117] Attention Mechanism
[0118] At this layer, the attention mechanism extracts important sentiment words related to the sentiment polarity of aspect words from the sentence context, and sets attention weights for each context word. The importance of each word varies, and the attention weights are calculated as follows:
[0119]
[0120] By performing a dot product between the aspect word representation and the context representation in the sentence, we can derive the correlation between the aspect word and other words in the sentence. The final representation of the aspect word is as follows:
[0121]
[0122] Sentiment Analysis
[0123] After obtaining the representation r, it is input into the fully connected layer and the softmax function to obtain the probability p(a) of aspect word a in various sentiment polarities:
[0124] p(a)=softmax(W p r+b p ), W p and b p These are all learnable parameters.
[0125] Model Training
[0126] The loss function used in this model is the cross entropy loss function:
[0127]
[0128] N represents the number of sentences, n represents the number of words in a sentence, θ represents all trainable parameters, and λ is the L2 regularization coefficient.
[0129] In order to capture the deep semantic information of the context and alleviate the overfitting problem, a novel random dense residual link is designed for the stacked BiLSTM. In addition, in order to take into account multiple relational features in the sentence, a multi-channel graph convolutional neural network is built, and a channel is designed for each relational feature. Unlike previous studies, the SE-MBGCN model can not only make full use of the deep semantic features of the context, but also integrate multiple relational features in the sentence into aspects to enrich the representation of aspects. Extensive experiments are also conducted on three public benchmark datasets. The experimental results show that the model has achieved state-of-the-art performance.
[0130] In future work, on the one hand, SE-MBGCN will be applied to other specific scenarios of natural language processing. On the other hand, the attention mechanism will be further simplified through part-of-speech relations to reduce noise and interference, so as to improve the overall performance of the model.
[0131] The above are only preferred embodiments of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.
Claims
1. A semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis, characterized by: include: Model training: By learning and training the data, the system learns key features and rules from the data and applies them to new training data; Sentiment analysis: analyzing the probability of words in various sentiment polarities; Attention mechanism: extract important sentiment words related to the sentiment polarity of aspect words from the sentence context, and set attention weights for each context word; Random dense residual connection BiLSTM: mining temporal information and semantic information in data; Word embedding: converting words in a sentence into vectorized representations; Mask layer: mask the non-aspect words in the sentence and only retain the vector representation of the aspect words; Multi-channel bidirectional graph convolutional network: used to better capture the features in the input data; The random dense residual connection BiLSTM includes a stacked BiLSTM and a random dense residual connection, and a bidirectional signal connection is formed between the stacked BiLSTM and the random dense residual connection. The multi-channel bidirectional graph convolution network includes a syntactic dependency tree, a graph convolution network, a multi-channel graph convolution network, a bidirectional graph convolution network, and a residual connection. The syntactic dependency tree, the graph convolution network, the multi-channel graph convolution network, the bidirectional graph convolution network, and the residual connection are all bidirectionally signal connected. The output end of the random dense residual connection in the random dense residual connection BiLSTM is bidirectionally connected to the input end of the model training, and the output end of the residual connection in the multi-channel bidirectional graph convolutional network is bidirectionally connected to the input end of the model training; Stacked BiLSTM: c represents the number of BiLSTM layers. After encoding the word vector using stacked BiLSTM, the sentence context representation H is obtained. c for: Recursively pass the representation of each layer to all subsequent layers, then linearly superimpose the hidden state of the subsequent layer and the hidden states of all its previous layers to represent the input of the next layer, and randomly discard dense residual connections during training; The original dense residual connection of the stacked BiLSTM has the following form: Each layer of dense residual connection adds a random parameter r c , r of each layer c Deciding to remove or retain dense residual connections and adding randomization parameters is as follows: Random parameter r c The value can only be 0 or 1. c = 0, the corresponding dense residual connection is deleted. c = 1, the corresponding dense residual connection is retained, r c is generated by Bernoulli distribution; The higher the BiLSTM layer, the more important the extracted semantic features are, and the greater the weight they occupy in the final output. Therefore, a weight parameter W is set for each layer. c ,The higher the BiLSTM layer, the larger the corresponding weight parameter, W c The representation is as follows: c represents the cth layer, C is the total number of layers, q is the global parameter, and the weight coefficient W is added c After that, it looks like this:
2. The semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis according to claim 1, characterized in that: The output signal of the word embedding is connected to the input of the random dense residual connection in the random dense residual connection BiLSTM.
3. The semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis according to claim 1, characterized in that: The output end of the word embedding is bidirectionally signal-connected to the input end of the model training.
4. The semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis according to claim 1, characterized in that: The output signal of the random dense residual connection in the random dense residual connection BiLSTM is connected to the input of the attention mechanism.
5. The semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis according to claim 1, characterized in that: Random Dense Residual Connection The output signal of the random dense residual connection in BiLSTM is connected to the input of the mask layer.
6. The semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis according to claim 1, characterized in that: The output end of the attention mechanism is bidirectionally connected to the input end of the model training.
7. The semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis according to claim 1, characterized in that: The output signal of the attention mechanism is connected to the input of the sentiment analysis.
8. The semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis according to claim 1, characterized in that: The output signal of the residual connection in the multi-channel bidirectional graph convolutional network is connected to the input of the mask layer.
9. The semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis according to claim 1, characterized in that: The output end of the mask layer is bidirectionally connected to the input end of the model training.
10. The semantically enhanced multi-channel bidirectional graph convolutional network system for aspect-level sentiment analysis according to claim 1, characterized in that: The output end of the mask layer is bidirectionally signal-connected to the input end of the attention mechanism.
