An aspect-level sentiment analysis method based on multi-level knowledge enhancement

Abstract

The application discloses a kind of aspect-level sentiment analysis method based on multilevel knowledge enhancement, belong to sentiment analysis field. Including the following steps: S1: text word vector is obtained using GloVe word embedding tool, the syntax dependency tree of sentence is constructed using Stanza, and the dependency graph is constructed accordingly;S2: the context representation of sentence is extracted by inputting word vector into BiLSTM, and the dependency graph is updated using sentiment dictionary and sensitive relationship set, to realize the sentiment and syntax enhancement of sentence;S3: the enhanced dependency graph is input into GCN to model node features, to obtain specific aspect representation;S4: aspect word is enhanced using concept atlas to obtain aspect word representation, which is fused with specific aspect representation to obtain aspect representation;S5: aspect representation and context representation are coordinated and optimized using interactive attention, to obtain the final representation of sentence, so as to determine aspect sentiment tendency.The application can effectively improve the accuracy of specific aspect sentiment classification, and help merchants accurately locate problems in products or services.

Description

Technical Field

[0001] This invention relates to the field of sentiment analysis technology, specifically an aspect-level sentiment analysis method based on multi-level knowledge enhancement. Background Technology

[0002] In recent years, the rapid development of information technology has had a significant impact on people's daily lives and work. Social media and online shopping platforms have generated a large number of valuable comments on people, events, and products. The process of analyzing, processing, and summarizing subjective texts containing emotional color is the task of sentiment analysis. Depending on the granularity of the text, sentiment analysis can be divided into coarse-grained and fine-grained sentiment analysis. Coarse-grained sentiment analysis mainly provides corresponding sentiment tendencies for texts at the discourse and sentence levels, while fine-grained sentiment analysis, also known as aspect-level sentiment analysis, aims to provide the sentiment tendencies of specific aspects within a sentence. For example, in the sentence "The food tastes very good, but the service needs improvement," the consumer gives a positive evaluation of the "food," containing positive emotion; however, for the "service," the consumer's comment is "needs improvement," containing negative emotion. Aspect-level sentiment analysis aims to provide the sentiment tendencies of various aspects within a comment statement.

[0003] Currently, most methods for aspect-level sentiment analysis are based on deep learning. Because comment texts are short and contain insufficient semantic information, many methods are beginning to incorporate external knowledge into the model to enrich the text's semantic information and improve sentiment classification accuracy. However, existing aspect-level sentiment analysis methods fail to comprehensively consider the impact of sentiment knowledge, conceptual knowledge, and syntactic dependencies between words on the accuracy of aspect sentiment judgment, resulting in poor parsing ability and low sentiment classification accuracy.

[0004] To address the above shortcomings, this invention proposes an aspect-level sentiment analysis method based on multi-layered knowledge enhancement. The key difference lies in enhancing the sentence dependency graph by embedding sentiment and syntactic knowledge into the comment statement, and learning node features through a graph convolutional network. Simultaneously, it utilizes Microsoft Concept Graph to enhance the conceptual representation of aspect words, achieving knowledge enhancement at the sentiment, syntax, and conceptual levels of the comment statement. This enriches the semantic representation of the text and improves the accuracy of aspect sentiment polarity discrimination. Summary of the Invention

[0005] The technical problem solved by this invention is to address the shortcomings of existing technologies by proposing an aspect-level sentiment analysis method based on multi-level knowledge enhancement. This method enriches the deep semantic information of the text by enhancing the comment statements with knowledge at three levels: sentiment, syntax, and concept, thereby improving the semantic parsing ability of the model and enhancing the accuracy of sentiment analysis.

[0006] To achieve the above objectives, the present invention provides an aspect-level sentiment analysis method based on multi-level knowledge enhancement, which is carried out according to the following steps:

[0007] Step 1: After data preprocessing and word segmentation, word embedding is performed to obtain text word vectors, and a dependency graph is constructed based on the sentence's syntactic dependency tree. The basic steps are as follows:

[0008] Step 1.1: After cleaning the data, segment the given sentence S into S = {w1, w2, w3, ..., w...} a1 ,w a2 ,...,w ak ,...,w n}, the sentence includes a sequence of aspect words of length k, a = {w a1 w a2 ,...,w ak}, and the aspect word sequence a is a subsequence of sentence S. Where w i w represents the i-th word in the context. ai This represents the i-th aspect term.

[0009] Step 1.2: Use the GloVe word embedding matrix W∈R de×V The words in a sentence are embedded into vectors of a specified dimension, resulting in the sentence embedding matrix E = {ew1,ew2,...,ew...} a1 ,ew a2 ,...,ew ak ,...,ew n}, where de is the word dimension, V is the total number of words, and ew i ∈R de It is the word embedding representation of the i-th context word, ew ai ∈R de It is the word embedding representation of the i-th aspect word.

[0010] Step 1.3: Use the Stanza natural language parsing tool to obtain the syntactic dependency tree of the comment statement, and construct the adjacency matrix A∈R based on the syntactic dependency tree. n×n , where n represents the number of words in the comment statement. The adjacency matrix is ​​constructed according to the following rules:

[0011]

[0012] Since directed graphs only allow unidirectional communication and lose some dependency information, this paper constructs an undirected graph to preserve the node information of each word. The resulting adjacency matrix has all 1s on the diagonal, i.e., A i,j =A j,i =1.

[0013] Step 2: Extract the contextual representation of the sentence and perform sentiment and syntactic enhancement on the comment statement. The basic steps are as follows:

[0014] BiLSTM can simultaneously capture both forward and reverse contextual information in text, deeply extracting semantic features from both directions of the text.

[0015] Step 2.1: Feed the sentence initialization vector E obtained after word embedding into BiLSTM to learn the hidden information of words in the sentence and obtain the contextual hidden representation of the sentence.

[0016]

[0017]

[0018] By concatenating the forward and reverse sentence hidden representations, we obtain the complete context hiding representation. Where H c ={h1,h2,h3,...,h n}∈R n×2de This represents the context of a sentence.

[0019] Step 2.2: Construct the sentiment intensity adjacency matrix S using the SentimentNet sentiment dictionary. i,j This enhances the emotional connection between context words and aspect words.

[0020]

[0021] Among them, SenticNet(w i )∈[-1,1], representing the word w i In the corresponding sentiment scores in the dictionary, strongly negative words have sentiment scores close to -1, while strongly positive words tend to have sentiment scores close to +1. When SenticNet(w i When ) = 0, it represents the word w i The sentiment of a word is neutral or does not exist in the dictionary. Examples of words and their sentiment scores in SenticNet are given in Table 1.

[0022] Table 1 Examples of word sentiment scores in SenticNet

[0023]

[0024]

[0025] At the same time, in the ABSA task, all word-level dependencies should not be considered equally important. When judging aspect sentiment, the model should pay more attention to opinion words related to aspect words.

[0026] like Figure 3In the given example, the aspect term "service" has dependencies on both the words "poor" and "the," but the adjective "poor" is the opinion term of the aspect term, with a dependency type of 'nsubj', meaning "service" is the nominal subject of "poor." The dependency type between the aspect term and "the" is 'det', meaning "the" is merely a determiner of "service." Clearly, compared to 'det', the 'nsubj' dependency type is more important for determining the sentimental inclination of the aspect.

[0027] Step 2.3: This invention utilizes the sensitive relation set SDR' to mine the dependency types between words and constructs a syntactically enhanced adjacency matrix D. i,j Implement syntactic enhancement for comment statements:

[0028]

[0029] in, Indicates the type of dependency relationship between context terms and aspect terms, when At that time, it was considered that the context word w i With aspect word w ai The more important dependency type is between them, where SDR'={'amod','neg','advmod','ccomp','compound','nmod','nummod','nsubj'} is the expanded set of sensitive relations.

[0030] At the same time, to highlight the importance of the aspect words, and also according to w i ,w j Construct the adjacency matrix for whether the term is an aspect term:

[0031]

[0032] After the above calculations, the adjacency matrix with enhanced syntax and sentiment knowledge is obtained:

[0033] A i,j =S i,j ×(D i,j +Z i,j +1)

[0034] Step 3: Model the node features and mask the non-aspect representations. The basic steps are as follows:

[0035] Step 3.1: Input the obtained enhanced adjacency matrix into the Graph Convolutional Network (GCN) to model node features. The i-th node in the l-th GCN layer represents the update of its neighborhood hidden layer:

[0036]

[0037]

[0038] Among them W l and b l These are the weight matrix and the bias term matrix, respectively. The output of the previous layer GCN serves as the hidden state of the node's input in layer l. F(*) is the position transformation function, which models the context words close to the aspect words through position weighting to reduce the noise generated during the process.

[0039] Step 3.2: To highlight the salient features of aspect words and eliminate noise from other states, the non-aspect representations in the output vector of the GCN layer are masked while keeping the aspect representations unchanged. The specific masking method is as follows:

[0040]

[0041] Where a1 and a m These represent the beginning and end of the aspect sequence, respectively. After masking, a specific aspect representation with enhanced sentiment and syntax is obtained:

[0042]

[0043] Step 4: After conceptual enhancement of the aspect terms, integrate them with specific aspect representations. The basic steps are as follows:

[0044] Step 4.1: Use Microsoft Concept Graph to conceptualize the terms, resulting in a concept set C = {c1, c2, ..., c...} n}, which contains n conceptual data, c i For the i-th concept in the concept set, the most suitable concept vector for each aspect word is calculated using a self-attention mechanism. The calculation process is as follows:

[0045]

[0046] Where W is the weight matrix, b is the bias term, V is the weight vector, and α i The attention score obtained by the i-th concept in the concept set is represented by the final concept vector of the aspect words after weighted calculation:

[0047]

[0048] Step 4.2: Concatenate the concept vector T with the aspect vector a to obtain the concept-enhanced aspect representation Ta = {Ta1, Ta2, ..., Ta...} k}, representing it with specific aspects enhanced by emotion and syntax H MaskAfter splicing, we obtain aspect representations H that are enhanced with knowledge at three levels: emotion, syntax, and concept. a =[Ta,H Mask ].

[0049] Step 5: Integrate aspect representations and contextual representations for classification to obtain the sentiment tendency of each aspect. The basic steps are as follows:

[0050] Step 5.1: Use an interactive attention mechanism to coordinate and optimize the obtained contextual representation and aspect-specific representation, H a H represents specific aspects of multi-level knowledge enhancement. c This represents the contextual representation of the Bi-LSTM output. The aspect-context attention calculation is as follows:

[0051]

[0052]

[0053]

[0054] By calculating the semantic association between the two, a corresponding attention weight α is assigned to each context word. i Thus, the context representation c is obtained. r Similarly, the aspect representation a can be calculated. r .

[0055] The context is represented as c r and aspect represent a r The final representation of the text obtained after fusion is r = [c r ,a r ].

[0056] Step 5.2: Use the final output r of the interaction attention layer as the input of the fully connected layer, and obtain the final emotional polarity y through softmax.

[0057] Beneficial effects

[0058] The method of the present invention has the following advantages compared with the prior art:

[0059] This method addresses the issue of insufficient semantic information provided by the comment statement itself. It utilizes sentiment, syntax, and conceptual knowledge to perform multi-level knowledge enhancement on the sentence, thereby enriching the semantic features of the text. This approach can better model the sentence representation and improve the model's accuracy in judging the sentiment tendency of each aspect. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in this invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0061] Figure 1 This is an overall flowchart of the method described in this invention.

[0062] Figure 2 This is a detailed framework diagram of the method described in this invention.

[0063] Figure 3 Provide a syntactic dependency graph for the example sentences. Detailed Implementation

[0064] To more clearly illustrate the technical solutions of this invention or the prior art, the method of this invention will be further described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The following embodiments are only used to illustrate this invention and cannot be used to limit the scope of this invention.

[0065] The following is combined Figure 2 The embodiments provided by the present invention are described below.

[0066] like Figure 2 As shown, the present invention includes the following steps:

[0067] Step 1: Extract the context representation of the sentence.

[0068] In this specific embodiment, the segmented sentence is processed using the GloVe word embedding tool to obtain word embedding vectors, which are then fed into BiLSTM to model contextual features, resulting in the context representation H. c .

[0069] Step 2: Construct a syntactic dependency graph of the comment statements.

[0070] Specifically, in this embodiment, the Stanza natural language parsing tool is used to obtain the syntactic dependency tree of the sentence, and a dependency graph is constructed based on the obtained dependency tree.

[0071] Step 3: Enhance the sentence with emotion and syntax.

[0072] Specifically, in this embodiment, the SenticNet sentiment dictionary and the extended sensitive relation set are used to construct sentiment-enhanced and syntactic-enhanced adjacency matrices. Then, the sentiment-enhanced and syntactic-enhanced adjacency matrices are input into a graph convolutional neural network to extract node features. Furthermore, the output of the graph neural network is masked for non-aspect representations, that is, all non-aspect word representations are set to 0 while keeping aspect representations unchanged.

[0073] Step 4: Enhance the concepts of the terms.

[0074] Specifically, in this embodiment, Microsoft Concept Graph is used in conjunction with a self-attention mechanism to enhance the aspect term concept representation, thereby obtaining the enhanced aspect term representation.

[0075] Step 5: Integrate contextual representations and aspect representations.

[0076] Specifically, in this embodiment, the aspect representation enhanced by sentiment and syntax and the aspect word representation enhanced by concept are fused to obtain three levels of enhanced aspect representation. An interactive attention mechanism is then used to coordinate and optimize the aspect representation and the context representation to obtain the final sentence representation.

[0077] Step 6: Sentiment Classification.

[0078] In this specific embodiment, the final sentence representation is fed into a Softmax classifier to obtain the final sentiment tendency.

[0079] For those skilled in the art, the present invention is not limited to the details of the exemplary embodiments described above, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered exemplary and not restrictive in all respects. The scope of the invention is defined by the appended claims rather than the foregoing description, and therefore all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0080] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method of aspect-level sentiment analysis based on multi-level knowledge enhancement, characterized in that The method comprises the following steps: Step 1: Preprocessing data, segmenting sentences, using GloVe word embedding tool to obtain text word vectors, and using syntax analysis tool to obtain syntax dependency tree of the sentences, and constructing a dependency graph according to the syntax dependency tree; Step 2: input the text word vector into BiLSTM to extract the context representation of the sentence, and update the dependency graph using the SenticNet sentiment dictionary and sensitive relationship set, so as to realize the sentiment and syntax enhancement of the review sentence; including: step 2.1: initialize the vector of the sentence obtained after word embedding BiLSTM is sent to learn the hidden information of the sentence, and the context hidden representation of the sentence is obtained: concatenating the forward and reverse sentence hidden representations to obtain a complete contextual sentence hidden representation wherein ; Step 2.2: Sentiment enhancement of the review sentences is implemented using SenticNet sentiment lexicon, and the sentiment intensity adjacency matrix is obtained : wherein, , indicates the sentiment score of the word corresponding in the dictionary, strong negative word sentiment score close to -1, strong positive word sentiment score tends to +1, when , indicates that the sentiment of the word is neutral or does not exist in the dictionary; Step 2.3: Utilizing the sensitive relation set Constructing a syntax augmented adjacency matrix D i,j Implementing syntax augmentation of review sentences: in, Indicates the type of dependency relationship between context terms and aspect terms, when At that time, the context words are considered and aspect words The more important dependency type is between them; For the expanded set of sensitive relationships, and at the same time according to Construct the adjacency matrix for whether the term is an aspect term: After the above calculation, the adjacent matrix enhanced by syntax and sentiment knowledge is obtained: ; Step 3: inputting the enhanced dependency graph into a graph convolution network to model node features, and obtaining specific aspect representation after aspect masking; Step 4: using Microsoft Concept Graph to enhance the aspect words to obtain aspect word representation, fusing the aspect word representation with the specific aspect representation to obtain aspect representation enhanced at three levels of sentiment, syntax and concept; Step 5: adopting an interaction attention mechanism to coordinate and optimize the aspect representation and the context representation, obtaining a final sentence representation, and inputting the sentence representation into a classifier to obtain a specific aspect sentiment tendency.

2. The aspect-level sentiment analysis method based on multi-level knowledge enhancement according to claim 1, characterized in that In step 1, the following is included: Step 1.1: After data cleaning, the given sentence S is segmented into , where the sentence includes a sequence of aspect terms with length k , and the sequence of aspect terms is a sub-sequence of the sentence S , where represents the i th term in the context, represents the i th aspect term;​ Step 1.2: Using pre-trained GloVe word embedding matrix Embedding words in a sentence into a vector of specified dimensions to get a sentence embedding matrix where de is the dimension of the word, V is the total number of vocabulary, and are the word embedding representations of the th context word and aspect word, respectively; Step 1.3: Obtain the syntactic dependency tree of the sentence using the Stanza natural language parsing tool, and build an adjacency matrix according to the syntactic dependency tree , represents the number of words in the comment sentence, and the adjacency matrix construction rule is as follows: 。 3.The aspect-level sentiment analysis method based on multi-level knowledge enhancement according to claim 1, characterized in that In step 3, the following is included: Step 3.1: The resulting augmented adjacency matrix is input to a graph convolutional neural network modeling node features, the th node representation of the th GCN layer is updated by its neighborhood. wherein and are weight matrix and bias term matrix, respectively, is the output result of the previous layer GCN, as the hidden state of the node input in the layer, is a position conversion function, used to reduce the noise generated in the process; Step 3.2: masking the non-aspect representation in the GCN layer output vector, and keeping the aspect representation unchanged, and the masking method is as follows: wherein and respectively represent the beginning and end of the aspect sequence, and the specific aspect representation after masking and sentiment and syntax enhancement: 。 4. The aspect-level sentiment analysis method based on multi-level knowledge enhancement according to claim 1, characterized in that In step 4, the following is included: Step 4.1: Aspect word conceptualization using Microsoft knowledge graph, resulting in a set of concepts wherein the set of concepts comprises concept data, is a representation of the concept in the set of concepts, the most suitable concept vector for each aspect word is computed by a self-attention mechanism, the computation process being as follows: in W This is the weight matrix. b For bias terms, V It is a weight vector. The first in the concept set i The attention scores obtained from each concept are weighted to obtain the final concept vector for each aspect word: Step 4.2: Concatenate the concept vector T with the aspect vector After concatenation, the specific aspect representation enhanced by the concept is obtained, which is masked by the specific aspect representation enhanced by the sentiment and syntax After concatenation, the specific aspect representation enhanced by the knowledge of the three levels of sentiment, syntax and concept is obtained .

5. The aspect-level sentiment analysis method based on multi-level knowledge enhancement according to claim 1, characterized in that In step 5, the following is included: Step 5.1: Coordinate optimization of the resulting context representation with the specific aspect representation using an interactive attention mechanism, for the multi-level knowledge enhanced specific aspect representation, for the context representation output by the BiLSTM, aspect-context attention calculation: assigning respective attention weights to each context word by computing semantic associations of both , thereby obtaining a context representation , and aspect representation can be computed similarly ; representing the context and aspects fusing to obtain a final representation of the text ; Step 5.2: Final output of the interaction attention layer As the input of the fully connected layer, the final sentiment orientation probability is obtained by the softmax classifier, and the classification result is obtained.

Images