An aspect-level sentiment analysis method based on an interactive graph convolution network

Abstract

This invention discloses an aspect-level sentiment analysis method based on interactive graph convolutional networks (GCNs). The method includes preprocessing the sentiment analysis dataset, embedding text sentences into vector representations, capturing sentence-aspect dependencies using three independent BiLSTMs, constructing two distinct modules, and building three heterogeneous graphs based on different information within each module. The graph structure data is then input into the two GCNs, and a dual affine module is used as a bridge to exchange features extracted by the two GCNs. These features are then averaged and pooled. Finally, an attention mechanism is used to capture important features from both modules. This invention is applicable to sentences containing multiple aspect words where some aspect words do not contain explicit sentiment features. It can also be used to accurately capture the most important word among multiple aspect words, effectively addressing special cases in aspect-level sentiment analysis tasks.

Description

Technical Field

[0001] This invention relates to the field of aspect-level sentiment analysis technology, and in particular to an aspect-level sentiment analysis method based on interactive graph convolutional networks. Background Technology

[0002] Natural Language Processing (NLP) is a science that integrates linguistics, computer science, and mathematics. Aspect-level sentiment analysis (ABSA) is an important aspect of NLP. The goal of this task is to determine the sentiment polarity (e.g., positive, negative, or neutral) of a given aspect in a sentence. It is of great significance for various tasks, such as product recommendation, political stance analysis, medical texts, and public health emergencies. Compared to sentence-level sentiment classification, it can perform more granular sentiment analysis. Taking the sentence "The food was good, but the service was poor" as an example, "food" and "service" are two aspects, and users have expressed positive and negative sentiments towards them, respectively. Therefore, aspect-level sentiment analysis can accurately identify the user's attitude towards a particular aspect, rather than simply assigning a sentiment polarity to a sentence.

[0003] There are two main research methods for aspect-level sentiment analysis: traditional analysis methods and mainstream deep learning methods. Traditional aspect-level sentiment analysis methods are mainly based on machine learning, which requires a large number of manually labeled samples. The labeled dataset is then used as the training set and machine learning algorithms are used for training. This approach makes the performance of the model mainly dependent on the effectiveness of feature engineering. In contrast, early deep learning-based models used recurrent neural networks (RNNs) to improve the accuracy of sentiment analysis by utilizing syntactic structure information.

[0004] Currently, in the field of aspect-level sentiment analysis, recurrent neural networks, convolutional neural networks, and attention mechanisms are among the more effective deep learning models. However, these methods all have the following significant drawbacks:

[0005] (1) It is difficult to accurately capture aspects without explicit emotional words. When multiple aspect words appear in a sentence or one aspect word does not contain explicit emotional features around the word, the model may have difficulty accurately understanding the emotional features of that aspect, thus making an incorrect judgment. For example, for the sentence "the service was the only thing good about this restaurant", the sentence contains two aspects "service" and "restaurant". For the first aspect word "service", according to the meaning of the sentence, its emotional feature word "good" can be found obviously, thus judging a positive emotion. However, for the aspect word "restaurant", no explicit emotional word describing it was found. This may lead to focusing on the word "good", which is close to it but has nothing to do with it.

[0006] (2) It is difficult to capture the most critical word in a multi-word phrase. When an aspect word contains multiple words, it fails to accurately extract the key word that has the greatest impact on aspect sentiment. For example, for the sentence "the smaller size was a bonus because of space restructions", its aspect word is composed of multiple words. This makes it difficult for the model to capture the key word "size" and instead focus on the modifier "smaller". Therefore, it is easy to make an incorrect prediction of negative sentiment.

[0007] In summary, existing aspect-level sentiment analysis methods struggle to accurately capture aspects without explicit sentiment words and the most critical words in multi-word phrases. Therefore, this invention proposes an aspect-level sentiment analysis method based on interactive graph convolutional networks to address the problems existing in the prior art. Summary of the Invention

[0008] To address the aforementioned problems, the present invention aims to propose an aspect-level sentiment analysis method based on interactive graph convolutional networks, which solves the problems that existing aspect-level sentiment analysis methods have difficulty accurately capturing aspects without explicit sentiment words, and have difficulty capturing the most critical words in multi-word phrases.

[0009] To achieve the objectives of this invention, the invention is implemented through the following technical solution: an aspect-level sentiment analysis method based on interactive graph convolutional networks, comprising the following steps:

[0010] Step 1: First, preprocess the sentiment analysis dataset to be analyzed, based on a length of... m Sentences S and a containing n Aspects of each word TTo construct a sentence aspect pair ( S , T );

[0011] Step 2: Use Bi-LSTM to extract the context information and aspect information of each word in the input sentence aspect, generate hidden state vectors for the context, and use three independent Bi-LSTMs for specific aspect modules and between aspect modules respectively;

[0012] Step 3: Construct a context graph based on context information and a syntax graph based on syntactic dependency tree in the specific aspect module as an adjacency matrix graph, and construct a knowledge graph based on external knowledge in the modules between aspects as an adjacency matrix graph;

[0013] Step 4: Construct an aspect-specific GCN in the aspect-specific module and an aspect-between GCN in the aspect-between module. Then, input the two constructed adjacency matrix graphs into the two GCNs respectively to obtain the output features of the aspect-specific module and the aspect-between module. and ;

[0014] Step 5: Use the dual affine module to interactively extract information from the outputs of the two GCNs to obtain... and ;

[0015] Step 6: Use mean pooling to perform average pooling on the output information obtained in Step 5, and use a retrieval-based attention mechanism to retrieve the attention score for each aspect, thus obtaining the final sentiment classification.

[0016] A further improvement is that, in step one, the data preprocessing of the sentiment analysis dataset specifically includes: removing numbers from the data, removing special symbols from the data, removing links from the data, and removing stop words from the data.

[0017] A further improvement is made in step three, where the adjacency matrix graph constructed between specific aspect modules and modules related to those aspects is represented as follows:

[0018]

[0019] in Represents a set of nodes. Indicates the associativity of edges. yes The adjacency matrix.

[0020] A further improvement lies in the following: In step three, the specific steps for constructing the adjacency matrix graph in the specific aspect module are as follows:

[0021] A1. In a specific aspect module, construct a context-based adjacency matrix graph to combine aspect words with other aspect words, aspect words with sentences, and the relationships between sentences. If an aspect word is in sentence S1 or sentence S2, it is represented by 1 in the corresponding position; otherwise, it is represented by 0.

[0022] A2. Use the spacy toolkit to generate the grammatical dependency tree of the sentence, construct a grammar-based adjacency matrix graph, and use PMI in the adjacency matrix to represent the dependency relationship between each node in the sentence. When PMI is positive, it indicates that the semantic relationship between words is high, and when PMI is negative, it indicates that the relevance between words is low or there is no relevance.

[0023] A3. Integrate the constructed context-based adjacency matrix graph and the syntax-based adjacency matrix graph to obtain a context-syntax-based adjacency matrix graph.

[0024] A further improvement is made in the following steps: In step three, the specific steps for constructing the adjacency matrix graph in the inter-aspect module are as follows: In the inter-aspect module, the SenceNet6 knowledge base is introduced, the related words of the words output by the SenceNet6 sentiment dictionary are used, and a knowledge graph is constructed as the adjacency matrix graph. The nodes in the knowledge graph consist of two types: one is the word nodes in the sentence, and the other is the knowledge nodes related to the word nodes obtained from the sentiment dictionary.

[0025] A further improvement lies in the following: In step four, the specific steps for inputting the adjacency matrix graph into the GCN are as follows:

[0026] B1. From the formula of GCN The input is the adjacency matrix A of the graph and the feature attributes of the nodes. The output is the feature attributes of the new node. ;

[0027] B2. Adjacency matrix for a specific aspect module Used to represent a normalized symmetric adjacency matrix. GCN indicates a specific aspect j The weight matrix of the layer, and express The degree matrix, , use The node output represents a specific aspect of the module;

[0028] B3. Adjacency matrix of modules between aspects Used to represent a normalized symmetric adjacency matrix. Indicates the GCN first cross-aspect module j The weight matrix of the layer, and express The degree matrix, , use This indicates the node output of modules between different aspects.

[0029] A further improvement lies in the following: In step five, the specific steps for extracting the output information using the dual affine module are as follows:

[0030] C1. Utilize a dual affine module to output features The input is fed into the Softmax function for calculation, and the calculation is performed through... ) ,in These are learnable parameters;

[0031] C2. Utilize a dual affine module to output features The input is fed into the Softmax function for calculation, and the calculation is performed through... ) ,in These are learnable parameters.

[0032] A further improvement lies in the following: In step six, the specific steps for performing mean pooling and the attention mechanism are as follows:

[0033] D1. Apply mean pooling to the nodes of specific aspect modules and modules between aspects to obtain the following results: and ;

[0034] D2. The obtained output and Performing a join operation yields r=[ , ];

[0035] D3. Use a retrieval-based attention mechanism to retrieve basic features related to aspect words from the hidden state vector and assign a retrieval-based attention weight to each context word.

[0036] The beneficial effects of this invention are as follows: This invention constructs different heterogeneous adjacency matrix graphs from multiple perspectives, such as contextual information related to sentences, grammatical dependency trees, and common sense knowledge, through GCN. These heterogeneous graphs are then combined to better integrate information from multiple aspects. This provides a significant advantage for aspects that do not contain explicit sentiment words or contain multiple words. Furthermore, this invention integrates various information through heterogeneous graphs. Unlike previous methods that simply used 0 and 1 to represent adjacency matrices, this invention considers the dependencies between contextual word nodes and sentences when constructing adjacency matrix graphs. It also introduces an external knowledge base and combines it with contextual grammatical information, resulting in better performance when judging aspect words in special cases. This allows for accurate capture of aspects without explicit sentiment words and the most crucial words in multi-word phrases, thereby improving the overall accuracy of sentiment analysis. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic diagram of the aspect-level sentiment analysis method of the present invention;

[0039] Figure 2 This is a visual analysis diagram of a sentence containing aspect phrases with multiple words in an embodiment of the present invention;

[0040] Figure 3 This is a visual analysis diagram of sentences containing multiple aspects in an embodiment of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] See Figure 1 , Figure 2 , Figure 3 This embodiment provides an aspect-level sentiment analysis method based on interactive graph convolutional networks, including the following steps:

[0043] Step 1: First, perform data preprocessing on the sentiment analysis dataset to be analyzed, including removing numbers, special symbols, links, and stop words, and then construct a sentence aspect pair ( S , T ),in S Represents a length of m The sentence, T Indicates a containing n Aspects of each word;

[0044] Step 2: Use Bi-LSTM (Bidirectional Long Short-Term Memory) to extract the contextual and aspect information of each word in the input sentence. Three independent Bi-LSTMs are used in specific aspect modules and between aspect modules, and the forward operation of the LSTM is represented as... The reverse operation is represented as The hidden output of the sentence is obtained using Bi-LSTM. Indicates contextual information, Information indicating aspects;

[0045] Step 3: Construct adjacency matrix graphs for specific aspect modules and modules between aspects, as follows:

[0046]

[0047] in Represents a set of nodes. Indicates the associativity of edges. yes The adjacency matrix;

[0048] The specific steps for constructing the adjacency matrix graph in a specific aspect module are as follows:

[0049] A1. In a specific aspect module, construct a context-based adjacency matrix graph to combine aspect words with other aspect words, aspect words with sentences, and the relationships between sentences. If an aspect word is in sentence S1 or sentence S2, it is represented by 1 in the corresponding position; otherwise, it is represented by 0.

[0050] A2. Use the spacy toolkit to generate the grammatical dependency tree of the sentence, construct a grammar-based adjacency matrix graph, and use PMI in the adjacency matrix to represent the dependency relationship between each node in the sentence. When PMI is positive, it indicates that the semantic relationship between words is high, and when PMI is negative, it indicates that the relevance between words is low or there is no relevance.

[0051] A3. Integrate the constructed context-based adjacency matrix graph and the syntax-based adjacency matrix graph to obtain a context-syntax-based adjacency matrix graph.

[0052] The specific steps for constructing the adjacency matrix graph in the cross-aspect module are as follows: In the cross-aspect module, the SenceNet6 knowledge base is introduced, and the related words of the words output by the SenceNet6 sentiment dictionary are used to construct a knowledge graph as the adjacency matrix graph. The nodes in the knowledge graph consist of two types: one is the word node in the sentence, and the other is the knowledge node related to the word node obtained from the sentiment dictionary. If the node is in the sentence and has a related node in the knowledge base, it is represented as 1, otherwise it is 0. Finally, the new knowledge graph adjacency matrix is ​​input into the GCN of the cross-aspect module.

[0053] Step 4: Construct an aspect-specific GCN (Graph Convolutional Network) in the aspect-specific module and an inter-aspect GCN in the inter-aspect module. Then, input the two constructed adjacency matrix graphs into the two GCNs respectively to obtain the output features of the aspect-specific module and the inter-aspect module. and The specific steps are as follows:

[0054] B1. From the formula of GCN The input is the adjacency matrix A of the graph and the feature attributes of the nodes. The output is the feature attributes of the new node. ;

[0055] B2. Adjacency matrix for a specific aspect module Used to represent a normalized symmetric adjacency matrix. GCN indicates a specific aspect j The weight matrix of the layer, and express The degree matrix, , use The node output represents a specific aspect of the module;

[0056] B3. Adjacency matrix of modules between aspects Used to represent a normalized symmetric adjacency matrix. Indicates the GCN first cross-aspect module j The weight matrix of the layer, and express The degree matrix, , use The node outputs of modules between aspects;

[0057] Step 5: Use the dual affine module to interactively extract information from the outputs of the two GCNs to obtain... and The specific steps are as follows:

[0058] C1. Utilize a dual affine module to output features The input is fed into the Softmax function for calculation, and the calculation is performed through... ) ,in These are learnable parameters;

[0059] C2. Utilize a dual affine module to output features The input is fed into the Softmax function for calculation, and the calculation is performed through... ) ,in These are learnable parameters;

[0060] Step Six: Perform average pooling on the output information obtained in Step Five, and then use a retrieval-based attention mechanism for retrieval. The specific steps are as follows:

[0061] D1. Apply mean pooling to the nodes of specific aspect modules and modules between aspects to obtain the following results: and ;

[0062] D2. The obtained output and Performing a join operation yields r=[ , ];

[0063] D3. Use a retrieval-based attention mechanism to retrieve basic features related to aspect words from the hidden state vector, and assign a retrieval-based attention weight to each context word.

[0064] Finally, attention scores for each aspect are obtained, leading to the final sentiment classification.

[0065] In this embodiment, specific aspect modules and modules between aspects together constitute the analysis model. The model's performance is mainly evaluated using commonly used metrics in sentiment analysis: accuracy and F1 score, a classification task performance metric based on the confusion matrix, as shown in Table 1 below.

[0066] Table 1

[0067]

[0068] in:

[0069] True Positive (TP): The number of positive classes predicted as positive (True Positive Rate)

[0070] True Negative (TN): The number of negative classes predicted as negative classes (true negative rate)

[0071] False Positive (FP): The number of negative classes predicted as positive (false positive rate)

[0072] False Negative (FN): The number of positive classes predicted as negative classes (false negative rate).

[0073] Accuracy is calculated by dividing the number of correctly classified samples by the total number of samples. A higher accuracy indicates a better classifier. The expression is:

[0074] Accuracy =

[0075] Recall refers to the probability that a sample that is actually positive will be classified as positive by the model. The expression is:

[0076] Recall rate =

[0077] F1 is the harmonic mean of precision and recall, expressed as:

[0078] F1=

[0079] This embodiment compares and analyzes four typical datasets to verify the proposed analysis method. Table 2 below shows the performance comparison of this embodiment with four other aspect-level sentiment analysis methods on several datasets.

[0080] Table 2

[0081]

[0082] This embodiment selects four different types of baseline models, which use syntactic information, context information, knowledge information, and other methods individually. This allows for a more comprehensive demonstration of the effectiveness of combining multiple methods in this invention. Table 2 selects a representative method with better performance from the four types of methods for comparison with the technology proposed in this invention. The results show that:

[0083] Although the analysis model in this embodiment has a 0.23% lower F1 score than the InterGCN model on the Lap14 dataset, its accuracy is improved by 3.07%. Furthermore, its performance on the other three datasets is generally better than other models based on syntax, context, and knowledge. This fully verifies the effectiveness of the analysis method of this invention, indicating that by starting from multiple perspectives and interactively integrating information from context, syntax, and knowledge, the performance of sentiment analysis can be further improved.

[0084] On the Rest14 dataset, where sentences containing multi-faceted words account for 36.58%, SI-GCN improves the Acc and F1 scores by 1.52% and 2.03% respectively compared to the best baseline model, Intergcn. This demonstrates that the present invention can indeed achieve better results for sentences containing multi-faceted words, and also proves the importance of the present invention's interactive extraction of specific aspect modules and cross-faceted modules.

[0085] Furthermore, to illustrate that the present invention can solve the problems that previous methods have not addressed, this embodiment selected two typical examples for visual analysis, and the results are as follows. Figure 2 and Figure 3 As shown, the darker the color, the higher the attention the method pays to that word.

[0086] exist Figure 2 In the sentence "The servers behind the counter are always friendly," the aspect term "servers behind the counter" is composed of multiple words. The analysis method proposed in this invention can accurately locate the important word "servers," while ignoring the phrase "behind the counter" which modifies it as a preposition. This demonstrates that the technology of this invention can accurately identify aspect terms and effectively recognize the sentiment contained in multiple words.

[0087] exist Figure 3 In the sentence "The service was the only thing good about this restaurant," there are two aspect words: "servers" and "restaurant." However, the aspect word "servers" has a very clear emotional polarity described by the word "good," while the aspect word "restaurant" does not have any clear emotional indicators. However, using the technique of this invention, it is easy to accurately infer the emotional polarity of the sentence without a clear emotional polarity based on the sentence and another aspect word containing a clear emotional polarity. This also demonstrates the superior performance of the method of this invention in effectively extracting key points and other aspects without clear emotional polarity words.

[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An aspect-level sentiment analysis method based on interactive graph convolutional networks, characterized in that, Includes the following steps: Step 1: First, preprocess the sentiment analysis dataset to be analyzed, based on a length of... m Sentences S and a containing n Aspects of each word T To construct a sentence aspect pair ( S , T ); Step 2: Use Bi-LSTM to extract the context information and aspect information of each word in the input sentence aspect, generate hidden state vectors for the context, and use three independent Bi-LSTMs for specific aspect modules and between aspect modules respectively; Step 3: Construct a context graph based on context information and a syntax graph based on syntactic dependency tree in the specific aspect module as an adjacency matrix graph, and construct a knowledge graph based on external knowledge in the modules between aspects as an adjacency matrix graph; Step 4: Construct an aspect-specific GCN in the aspect-specific module and an aspect-between GCN in the aspect-between module. Then, input the two constructed adjacency matrix graphs into the two GCNs respectively to obtain the output features of the aspect-specific module and the aspect-between module. and The specific steps for inputting the adjacency matrix graph into the GCN are as follows: B1. From the formula of GCN The input is the adjacency matrix A of the graph and the feature attributes of the nodes. The output is the feature attributes of the new node. ; B2. Adjacency matrix for a specific module Used to represent a normalized symmetric adjacency matrix. GCN indicates a specific aspect j The weight matrix of the layer, and express The degree matrix, , use The node output represents a specific aspect of the module; B3. Adjacency matrix of modules between aspects Used to represent a normalized symmetric adjacency matrix. Indicates the GCN first cross-aspect module j The weight matrix of the layer, and express The degree matrix, , use The node outputs of modules between aspects; Step 5: Use the dual affine module to interactively extract information from the outputs of the two GCNs to obtain... and ; Step 6: Use mean pooling to perform average pooling on the output information obtained in Step 5, and use a retrieval-based attention mechanism to retrieve the attention score for each aspect, thus obtaining the final sentiment classification.

2. The aspect-level sentiment analysis method based on interactive graph convolutional networks according to claim 1, characterized in that: In step one, the data preprocessing of the sentiment analysis dataset specifically includes: removing numbers from the data, removing special symbols from the data, removing links from the data, and removing stop words from the data.

3. The aspect-level sentiment analysis method based on interactive graph convolutional networks according to claim 1, characterized in that: In step three, the adjacency matrix graph constructed between modules of a specific aspect and between aspects is represented as follows: in Represents a set of nodes. Indicates the associativity of edges. yes The adjacency matrix.

4. The aspect-level sentiment analysis method based on interactive graph convolutional networks according to claim 1, characterized in that: In step three, the specific steps for constructing the adjacency matrix graph in the specific aspect module are as follows: A1. In a specific aspect module, construct a context-based adjacency matrix graph to combine aspect words with other aspect words, aspect words with sentences, and the relationships between sentences. If an aspect word is in sentence S1 or sentence S2, it is represented by 1 in the corresponding position; otherwise, it is represented by 0. A2. Use the spacy toolkit to generate the grammatical dependency tree of the sentence, construct a grammar-based adjacency matrix graph, and use PMI in the adjacency matrix to represent the dependency relationship between each node in the sentence. When PMI is positive, it indicates that the semantic relationship between words is high, and when PMI is negative, it indicates that the relevance between words is low or there is no relevance. A3. Integrate the constructed context-based adjacency matrix graph and the syntax-based adjacency matrix graph to obtain a context-syntax-based adjacency matrix graph.

5. The aspect-level sentiment analysis method based on interactive graph convolutional networks according to claim 1, characterized in that: In step three, the specific steps for constructing the adjacency matrix graph in the inter-aspect module are as follows: In the inter-aspect module, the SensenceNet6 knowledge base is introduced, the related words of the words output by the SensenceNet6 sentiment dictionary are used, and a knowledge graph is constructed as the adjacency matrix graph. The nodes in the knowledge graph consist of two types: one is the word nodes in the sentence, and the other is the knowledge nodes related to the word nodes obtained from the sentiment dictionary.

6. The aspect-level sentiment analysis method based on interactive graph convolutional networks according to claim 1, characterized in that: In step five, the specific steps for extracting the output information using the dual affine module are as follows: C1. Utilize a dual affine module to output features The input is fed into the Softmax function for calculation, and the calculation is performed through... ) ,in These are learnable parameters; C2. Utilize a dual affine module to output features The input is fed into the Softmax function for calculation, and the calculation is performed through... ) ,in These are learnable parameters.

7. The aspect-level sentiment analysis method based on interactive graph convolutional networks according to claim 1, characterized in that: In step six, the specific steps for performing mean pooling and the attention mechanism are as follows: D1. Apply mean pooling to the nodes of specific aspect modules and modules between aspects to obtain the following results: and ; D2. The obtained output and Performing a join operation yields r=[ , ]; D3. Use a retrieval-based attention mechanism to retrieve basic features related to aspect words from the hidden state vector and assign a retrieval-based attention weight to each context word.

Images