An Aspect-Level Sentiment Analysis Method Based on Graph Convolutional Network

Abstract

The present invention belongs to the field of natural language processing, and discloses an aspect-level sentiment analysis method based on a graph convolutional network. In this method, Chinese review texts are encoded by BERT to obtain word-level text sequence vector representations. Then, the vector representations are used to obtain the syntactic dependency tree of the sentence through CDT to get the adjacency matrix. Next, the syntactic distance and the syntactic distance weight are calculated according to the syntactic dependency tree. Using the graph convolutional neural network, an accurate aspect word vector representation is obtained by combining the syntactic distance weight. The conjunctions between multi-aspect words are found, and a weight matrix is obtained from the conjunctions and multi-aspect words. The weight matrix, conjunctions, and aspect words are passed through the graph convolutional network to obtain the vector representation of the final aspect word. The vector representation is sent to the linear representation layer to obtain the final prediction result. Through the graph convolutional network, the present invention strengthens the relationship between aspect words and other words, effectively improving the accuracy of aspect-level sentiment analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and more specifically, to a Chinese aspect-level sentiment analysis method based on a graph convolutional network. Background Art

[0002] Sentiment analysis, also known as opinion mining, is a branch of natural language processing (NLP) that mainly studies the opinions and emotions expressed by people towards a certain event or product. The obtained sentiment polarities are generally divided into three categories: positive, neutral, and negative. With the advent of the Internet era, sentiment analysis has been widely applied in product recommendation, healthcare, and social events.

[0003] From the perspective of the fine-grainedness of sentiment analysis, sentiment analysis can be further divided into two categories. One is to perform sentiment analysis on a text passage or an entire sentence, and the other is to perform data mining on the attributes and evaluation objects contained in a sentence. In actual application scenarios, people often need to identify the objects expressed or evaluated by opinions or emotions, as well as the specific opinion tendencies expressed towards these objects. At this time, relying solely on sentiment analysis is not enough. For example, in the sentence "The food in the restaurant is delicious, but the environment is poor", in this sentence, the restaurant wants to obtain the opinions of the user on the food and the environment. Traditional sentiment analysis only outputs a single sentiment polarity for the entire sentence. Since the sentiment polarity corresponding to the food is positive and the sentiment polarity corresponding to the environment is negative, and the opinions expressed by the two are completely opposite, it is impossible to achieve this only by relying on traditional sentiment analysis.

[0004] Moreover, for the massive user product reviews on e-commerce platforms, the existing aspect-level sentiment analysis methods do not consider the influence of distance on aspect words and ignore the role of conjunctions between aspect words, resulting in low accuracy in sentiment analysis of aspect words. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides an aspect-level sentiment analysis method based on a graph convolutional network. This method simultaneously integrates the distance factor between aspect words and other words, as well as the conjunctions between multiple aspect words, and has a better improvement in prediction accuracy compared with existing algorithms.

[0006] To achieve the above object, the present invention is realized through the following technical solutions:

[0007] The present invention is an aspect-level sentiment analysis method based on a graph convolutional network, and this method includes the following steps:

[0008] Step 1: Obtain text review data, including Chinese review text and evaluation categories;

[0009] Step 2: Obtain aspect words in the review text;

[0010] Step 3: Encode the Chinese review text through the pre-trained model BERT to obtain a word-level text sequence vector representation containing context semantic information. The word-level text sequence vector representation containing context semantic information includes word vectors of the target category and other relevant context word vectors;

[0011] Step 4: Obtain the syntactic dependency tree of the sentence through the dependency tree convolution model (CDT) for the obtained word-level text sequence vector representation, and obtain an adjacency matrix;

[0012] Step 5: Calculate the syntactic distance and syntactic distance weight according to the syntactic dependency tree;

[0013] Step 6: Use the adjacency matrix obtained from the syntactic dependency tree in Step 4 and the word-level text sequence vector representation as inputs, and utilize a graph convolutional neural network to combine with the syntactic distance weight calculated in Step 5 to obtain an accurate aspect word vector representation;

[0014] Step 7: Find conjunctions between multi-aspect words through the LCA algorithm and the established mapping relationship;

[0015] Step 8: Obtain a weight matrix for the conjunctions and multi-aspect words through self-attention;

[0016] Step 9: Pass the weight matrix obtained in Step 8, along with the conjunctions and aspect words, through a graph convolutional network to obtain the final vector representation of the aspect words;

[0017] Step 10: Feed the vector representation of the aspect words obtained in Step 8 into a linear representation layer to obtain the final prediction result.

[0018] Preferably: In Step 5, the syntactic distance and syntactic distance weight are obtained based on the syntactic dependency tree. The specific steps for calculating the syntactic distance are as follows: Take aspect word a j and word w i as inputs, and output the syntactic distance between aspect word a j and word w i ; Calculate the syntactic distances of aspect word a j and word w i relative to the root node of the syntactic dependency tree, denoted as num1 and num2. Denote the distance between a j and w i as First, take Add the absolute value of the difference between num1 and num2. If the two nodes are the same at this time, it means that the two words are on the same branch. Then the is the final result. Otherwise, it means that the two nodes are not on the same branch. Take min(num1, num2), add twice of min(num1, num2) to get the final result. The corresponding syntactic distance D i,a (d 1,a , d 2,a … d i,a ) can be calculated by the following formula for the syntactic distance weight:

[0019]

[0020]

[0021] where d max is the maximum value in D i,a , and the position weight s i ∈[0.5, 1]. On the one hand, it ensures that the minimum weight of the words far from the attribute in the sentence is within a certain range. On the other hand, it makes the words closer to the attribute have a higher weight, preventing the model from being misled by strong sentiment words irrelevant to the attribute when predicting the sentiment polarity of the attribute. If there are multiple aspect words in the sentence, perform a weighted sum on the obtained .

[0022] Preferably, the formula for training using the graph convolutional network in step 6 is as follows:

[0023]

[0024] F l = s i h i

[0025] where W l and b l represent the weight matrix and bias vector of the l-th layer of the graph convolutional network respectively, ρ represents the RELU activation function, and when l = 1, denotes the output of the $i$-th hidden layer state in the $(l - 1)$-th layer graph convolutional network. In the model, when using the graph convolutional network to extract features, the hidden layer state representation of the sentence context is not directly input into the graph convolutional network. Instead, based on the existing sentence context information representation, a syntactic distance weight vector is used to first weight the context information related to the aspect term in the sentence syntactically. That is, according to the distance between each word and the aspect term in the syntactic dependency tree, different syntactic distance weights are assigned to each word. Words with a closer syntactic distance to the aspect term have larger weights, while words with a farther syntactic distance to the aspect term have smaller weights. Using the syntactic distance weights can highlight the sentiment information related to the aspect term and reduce the influence of words unrelated to the aspect term on aspect-level sentiment analysis, further distinguishing the context information related to the aspect term syntactically. Among them, $s$ i denotes the syntactic distance weight of the $i$-th word in the sentence, and $h$ i denotes the output of the $i$-th hidden layer state, and $F$ l denotes the context representation after adding the syntactic distance. The context representation $F$ weighted by the syntactic distance l highlights the sentiment information related to the aspect.

[0026] Preferably: Step 7 designs a mapping relationship to find the conjunction between two aspect terms,

[0027]

[0028] where: $a$ i and $a$ j denote two adjacent aspect terms. The $Br$ function finds the conjunction between the two aspect terms. $\omega$ k is the $Br$ function. When no conjunction is found, all the word vectors between the two aspect terms are returned. The lowest common ancestor between the two aspect terms is found in the syntactic dependency tree through the LCA algorithm. If it does not exist, all the words between the two aspect terms are selected.

[0029] Preferably, the sentiment classification representation result in Step 10:

[0030] $y = Softmax(W_0r + b_0)$

[0031] where $W_0$ and $b_0$ are trainable parameters, $r$ is the hidden layer vector representation obtained in Step 9, and $y$ is the output sentiment classification result.

[0032] The beneficial effects of the present invention are:

[0033] (1) During the training process of using the graph convolutional network in the present invention, the distance weight between the aspect term and its context is combined for training, strengthening the relationship between the aspect term and its context;

[0034] (2) The present invention takes into account the situation where multi-aspect words appear in a sentence. For example, when calculating the distance weight, an average is taken, and the relationship between multi-aspect words is strengthened by finding the conjunctions of multi-aspect words;

[0035] (3) By using the attention mechanism for multi-aspect words and their conjunctions, the present invention strengthens the semantic relationship between multi-aspect words, making the vector representation of aspect words more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the syntactic dependency tree formed in the embodiment of the present invention.

[0037] Figure 2 It is a schematic diagram of the training process of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] The embodiments of the present invention will be disclosed below with reference to the drawings. For the sake of clarity, many practical details will be described together in the following description. However, it should be understood that these practical details are not used to limit the present invention. That is to say, in some embodiments of the present invention, these practical details are not necessary.

[0039] As Figure 1 shown, an aspect-level sentiment analysis method based on a graph convolutional network according to the present invention is characterized in that: the aspect-level sentiment analysis method includes the following steps:

[0040] Step 1, obtain text review data, the data includes: Chinese review text, evaluation category;

[0041] Step 2, obtain the aspect words in the review text;

[0042] Step 3, encode the Chinese review text through the pre-trained model BERT to obtain a word-level text sequence vector representation containing context semantic information, and the word-level text sequence vector representation containing context semantic information includes word vectors of the target category and other word vectors;

[0043] Step 4, obtain the syntactic dependency tree of the sentence through the dependency tree convolution model (CDT) for the obtained word-level text sequence vector representation, and obtain an adjacency matrix;

[0044] Step 5, calculate the syntactic distance and syntactic distance weight according to the syntactic dependency tree;

[0045] Step 6, using the adjacency matrix obtained from the syntactic dependency tree in Step 4 and the word-level text sequence vector representation as inputs, and using a graph convolutional neural network, combine the syntactic distance weight calculated in Step 5 to obtain an accurate aspect word vector representation;

[0046] Step 7: Find the conjunctions between multi-faceted words through the LCA algorithm and the established mapping relationship;

[0047] Step 8: Obtain the weight matrix for the conjunctions and multi-faceted words through self-attention;

[0048] Step 9: Pass the weight matrix obtained in Step 8, along with the conjunctions and aspect words, through a graph convolutional network to obtain the vector representation of the final aspect words;

[0049] Step 10: Feed the vector representation of the aspect words obtained in Step 8 into the linear representation layer to obtain the final prediction result.

[0050] Suppose the collected data includes:

[0051] “this lunch food taste very well,but the restaurant has poorservice.”

[0052] Encode this sentence through the pre-trained model BERT to obtain a word-level text sequence vector representation containing context semantic information. The word-level text sequence vector representation containing context semantic information includes word vectors of the target category and other word vectors.

[0053] Then, obtain the syntactic dependency tree of the sentence through the dependency tree convolutional model (CDT). The syntactic dependency tree has three uses. It can generate the adjacency matrix A of the first graph convolutional network and can calculate the distance weight S through the following algorithm.

[0054] In the graph convolutional network model, correct the vector representations of each word. Take the aspect word a j and the word w i as inputs, and output the syntactic distance between the aspect word a j and the word w i . First, calculate the syntactic distances of the aspect word a j and the word w i relative to the root node of the syntactic dependency tree, denoted as num1 and num2. Denote the distance between a i and w i as First, add the absolute value of the difference between num1 and num2 to . If the two nodes are the same at this time, it means that the two words are on the same branch. Then, at this time is the final result. Otherwise, it means that the two nodes are not on the same branch. Take min(num1, num2), add twice of min(num1, num2) to it to get the final result. The corresponding syntactic distance D can be obtained by the above algorithmi,a (d 1,a ,d 2,a …d i,a ), the syntactic distance weight is calculated using the following formula:

[0055]

[0056]

[0057] where d max is the maximum value in D i,a , the position weight s i ∈[0.5, 1]. On the one hand, it ensures that the minimum weight of words far from the attribute in the sentence is within a certain range. On the other hand, it makes the words closer to the attribute have a higher weight, preventing the model from being misled by strong sentiment words unrelated to the attribute when predicting the sentiment polarity of the attribute. If there are multiple aspect words in the sentence, the obtained is weighted and summed.

[0058] For example Figure 1 as described, taking the aspect words service and well as an example, the distance between service and the root node taste is num1 = 2, and the distance between well and taste is num2 = 1, num2 - num1 = 1. Regardless of whether service and well are in the same branch, this distance is the necessary path from service to the well node. It can be considered that the service node reaches the position of the has node at this time, and then check whether the service is at the same node as well at this time. If not, then a twice of min(num1, num2) needs to be added, which is exactly the distance from has to well, and the sum is 3; d max = 4, so for a1service, Similarly, for the aspect word taste, the distance of well is 1, so the weighted sum finally gets Through the above algorithm, finally, S(s1, s2,..., s n ) can be obtained.

[0059] Next, through the training of the graph convolutional network, the vector representation of the aspect words is obtained, and the training formula is as follows:

[0060]

[0061] F l = s i h i

[0062] where, W l and b lrespectively represent the weight matrix and bias vector of the l-th layer graph convolutional network, ρ represents the RELU activation function, and when l = 1, represents the output of the i-th hidden layer state in the (l - 1)-th layer graph convolutional network. In the model, when using the graph convolutional network to extract features, the hidden layer state representation of the sentence context is not directly input into the graph convolutional network. Instead, based on the existing sentence context information representation, using the syntactic distance weight vector, the context information syntactically related to the aspect term in the sentence is first weighted. That is, according to the distance between each word and the aspect term in the syntactic dependency tree, different syntactic distance weights are assigned to each word. Words with a closer syntactic distance to the aspect term have larger weights, while words with a farther syntactic distance to the aspect term have smaller weights. Using the syntactic distance weight can highlight the sentiment information related to the aspect term and reduce the influence of words unrelated to the aspect term on the aspect-level sentiment analysis, and further distinguish the context information syntactically related to the aspect term. Among them, s i represents the syntactic distance weight of the i-th word in the sentence, h i represents the output of the i-th hidden layer state, F l represents the context representation after adding the syntactic distance. The context representation F weighted by the syntactic distance l highlights the sentiment information related to the aspect.

[0063] On the syntactic dependency tree, the connector C between multi-aspect terms can be found through the LCA algorithm. The obtained C and the aspect term vector trained by the graph convolutional network are used as inputs. Through the graph convolutional network, the final aspect term vector representation is obtained. Finally, through the linear representation layer and the softmax layer, the final judgment result is obtained. service corresponds to negative, and taste corresponds to positive.

[0064] The sentiment analysis algorithm based on the graph neural network proposed by the present invention simultaneously integrates the distance factor and the conjunctions between various aspect terms, and has better performance in terms of recommendation accuracy, recall rate, and F value.

[0065] The above is only a partial implementation manner of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. An aspect-level sentiment analysis method based on graph convolutional network, characterized in that: The aspect-level sentiment analysis method includes the following steps: Step 1: Obtain text review data, including Chinese review text and evaluation categories; Step 2: Obtain aspect words in the review text; Step 3: Encode the Chinese review text through the pre-trained model BERT to obtain a word-level text sequence vector representation containing context semantic information. The word-level text sequence vector representation containing context semantic information includes word vectors of the target category and relevant context vectors; Step 4: Obtain the syntactic dependency tree of the sentence through the dependency tree convolution model (CDT) for the obtained word-level text sequence vector representation to obtain an adjacency matrix; Step 5: Calculate the syntactic distance and syntactic distance weight based on the syntactic dependency tree; Step 6: Use the adjacency matrix obtained from the syntactic dependency tree in Step 4 and the word-level text sequence vector representation as inputs, and use a graph convolutional neural network to combine the syntactic distance weight calculated in Step 5 to obtain an accurate aspect word vector representation; Step 7: Find the conjunctions between multi-aspect words through the LCA algorithm and the established mapping relationship; Step 8: Obtain a weight matrix for the conjunctions and multi-aspect words through self-attention; Step 9: Pass the weight matrix obtained in Step 8, as well as the conjunctions and aspect words, through a graph convolutional network to obtain the vector representation of the final aspect word; Step 10: Feed the vector representation of the aspect word obtained in Step 8 into a linear representation layer to obtain the final prediction result.

2. The aspect-level sentiment analysis method based on graph convolutional network according to claim 1, characterized in that: In Step 5, the syntactic distance and syntactic distance weight are obtained based on the syntactic dependency tree. The specific steps for calculating the syntactic distance are as follows: Step 5-1: Take aspect word a j and word w i as input, and output the syntactic distance between aspect word a j and word w i ; Step 5-2: Calculate aspect word a j and word w i 's syntactic dependency tree root node's grammatical distances, denoted as num1 and num2, and denote the distance between a j and w i as Step 5-3: First, add the absolute value of the difference between num1 and num2. If the two nodes are the same at this time, it means that the two words are on the same branch. Then is the final result. Otherwise, it means that the two nodes are not on the same branch. Take min(num1, num2), and add twice min(num1, num2) to get the final result. The corresponding syntactic distance D i,a (d 1,a , d 2,a … d i,a ) can be obtained by the above algorithm. The syntactic distance weight is calculated using the following formula: where d max is the maximum value of D i,a , and the position weight s i ∈[0.5, 1]. On the one hand, it ensures that the minimum weight of words far from the attribute in the sentence is within a certain range. On the other hand, it makes the words closer to the attribute have higher weights, preventing the model from being misled by strong sentiment words unrelated to the attribute when predicting the sentiment polarity of the attribute. If there are multiple aspect words in the sentence, the obtained weighted sum can be used directly.

3. The aspect-level sentiment analysis method based on graph convolutional network according to claim 1, characterized in that: In Step 6, the formula for training using a graph convolutional network is as follows: F l = s i h i Among them, W l and b l respectively represent the weight matrix and bias vector of the l-th layer graph convolutional network, ρ represents the ReLU activation function, and when l = 1, represents the output of the i-th hidden layer state in the (l - 1)-th layer graph convolutional network, s i represents the syntactic distance weight of the i-th word in the sentence, h i represents the output of the i-th hidden layer state, F l represents the context representation after adding the syntactic distance, and the context representation F weighted by the syntactic distance l highlights the aspect-related sentiment information.

4. The aspect-level sentiment analysis method based on graph convolutional network according to claim 1, characterized in that: In Step 7, an established mapping relationship is used to find the conjunction between two aspect words, which is expressed as: Where: a i and a j represent two adjacent aspect words. The Br function finds the conjunction between the two aspect words. ω k is the Br function. When no conjunction is found, all the word vectors between the two aspect words are returned. The lowest common ancestor between the two aspect words is found in the syntactic dependency tree through the LCA algorithm. If it does not exist, all the words between the two aspect words are selected.

5. The aspect-level sentiment analysis method based on graph convolutional network according to claim 1, characterized in that: In Step 10, the prediction result of sentiment classification is expressed as: y = Softmax(W0r + b0) where W0 and b0 are trainable parameters, r is the hidden layer vector representation obtained in Step 9, and y is the output sentiment classification result.

6. The method for Chinese aspect-level sentiment analysis based on graph neural network according to claim 1, wherein The evaluation categories in Step 1 include: location, catering, room facilities, entertainment facilities, public areas, and services.

Images