A method for aspect-level sentiment classification based on directed acyclic graph networks

Abstract

The present invention discloses an aspect-level sentiment classification method based on a directed acyclic graph network, which is characterized in that the method specifically includes the following steps: extracting text feature representation, obtaining a bidirectional directed acyclic graph using a grammatical dependency tree encoding, integrating word sentiment knowledge into the bidirectional directed acyclic graph, encoding text representation with grammatical dependency using a bidirectional directed acyclic graph neural network, utilizing an aspect-level masking mechanism for sentiment representation, integrating masked high-order sentiment representation, obtaining high-order aspect sentiment representation and applying linear changes to obtain a final sentiment classification. Compared with the prior art, the present invention has the ability to cope with sentiment analysis of complex long sentences in reality, and better solves the problem of obvious over-smoothing of the bidirectional directed acyclic graph neural network when stacking layers. By stacking network layers, the representation of each word is continuously iteratively optimized to obtain more efficient sentiment transmission. The method is simple, effective, and has good application prospects.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to an aspect-level sentiment classification method based on a directed acyclic graph network. Background Art

[0002] With the rapid development of digital technologies, a vast amount of opinionated material has been generated online. Individuals are more likely to express their opinions on several aspects of products, services, or reviews, and these opinions are often considered important for a wide range of applications, such as product review analysis, public opinion analysis, and personalized recommendations. However, the vast amount of online content makes learning the specific information about these reviews a cumbersome process. Consequently, aspect-based sentiment analysis (ABSA) has attracted increasing attention in the field of natural language processing (NLP), aiming to find practical and scalable solutions to automatically solve this task. Aspect-level sentiment classification is a fine-grained sentiment analysis task that aims to predict the sentiment polarity (positive, neutral, or negative) of a given aspect word.

[0003] Early research primarily used machine learning algorithms to construct sentiment classifiers. With the development of various neural networks, a series of neural network-based methods have been proposed and have achieved remarkable results. Recently, methods based on graph neural networks have achieved even better results. These methods use graph neural networks to encode the grammatical dependency tree of processed sentences, extracting grammatical dependencies within the sentences. These dependencies shorten the distance between aspect words and the corresponding sentiment words in the sentences, efficiently extracting the sentiment words corresponding to the aspect words, and thus better predicting the sentiment polarity of the aspect words. However, this method has the following issues:

[0004] The real world contains numerous complex and long sentences. For these sentences, the distance between aspect words and sentiment words is quite large, and even if grammatical dependencies are used to shorten them, the distance can still be quite long. To address this need for extracting high-order connections, it is necessary to stack layers of graph neural networks to extract long-range, high-order information. Due to the inherent characteristics of graph neural networks, as layers are stacked, the intersection of nodes perceived by each node increases, and the representations of each node become increasingly similar. This leads to a significant oversmoothing problem and a rapid decline in performance. Therefore, traditional graph neural encoding methods, dominated by graph convolutional networks (GCNs) and graph attention networks (GATs), do not effectively address this issue. Furthermore, while external sentiment knowledge has been shown to aid sentence sentiment perception, it has not been fully utilized to model aspect word sentiment in aspect-level sentiment classification tasks. Summary of the Invention

[0005] The purpose of the present invention is to propose an aspect-level sentiment classification method based on a directed acyclic graph network in response to the shortcomings of the existing technology. It adopts a bidirectional information transmission method in a simulated tree, constructs a directed acyclic graph neural network to propagate information of each node in the directed acyclic graph structure, simulates the bidirectional information flow in the grammatical dependency tree, and uses the sentiment information from SenticNet to enhance the sentiment expression ability of the bidirectional directed acyclic graph. The top-down and bottom-up directed acyclic graphs are successively input into the directed acyclic graph neural network to transmit sentiment information between words. Finally, the aspect mask is used to highlight the expression of aspect words, and the retrieval-based attention mechanism is used to further retrieve information from the context. The linear transformation is input to classify the sentiment polarity of the aspect words, effectively extract high-order information, and simulate the information flow in the grammatical dependency tree. A bidirectional directed acyclic graph is used to encode the grammatical dependency tree, simulate the bidirectional information transmission method in the tree, and add the emotional knowledge of each word. This emotion-enhanced bidirectional directed acyclic graph structure and the sequential node information transmission can extract high-order information when stacking fewer layers. It can cope with the emotional analysis of complex and long sentences in reality, and better solves the problem of obvious over-smoothing of bidirectional directed acyclic graph neural networks when stacking layers. The stacked network layers can be used to continuously iteratively optimize the representation of each word to obtain more efficient emotional transmission. The method is simple, effective, and has good application prospects.

[0006] The purpose of the present invention is achieved as follows: an aspect-level sentiment classification method based on a directed acyclic graph network, which is characterized in that the method first uses a large-scale pre-trained model to represent the input sentence, capture the semantic information in the sentence, and output the embedding matrix of the sentence; in order to add the grammatical connection between words, we encode the grammatical dependency tree from the grammatical parser into a bidirectional directed acyclic graph to simulate the transmission process of bidirectional grammatical dependency information in the sentence, and add the sentiment information of each word itself; then, the finally generated bidirectional directed acyclic graph is input into the subsequent bidirectional directed acyclic graph network to extract information; then, the output of the final layer of the bidirectional directed acyclic graph network is aspect-masked to highlight the expression of aspect words; finally, combined with the initial sentence semantic expression, the attention method is used for fusion, and the final score is obtained through linear transformation for corresponding classification. Specifically including the following steps:

[0007] 1) Input the vectorized word matrix into the bidirectional long short-term memory network to extract text feature representation, as follows:

[0008] 1.1: Input the sentence to be processed into the BERT pre-trained model to obtain the word vector of each word, combine them into a word vector matrix, and extract the semantic information in the sentence.

[0009] 1.2: The obtained word vector matrix is input into a bidirectional long short-term memory (LSTM) network for sequence modeling to obtain text feature representation.

[0010] 2) Use the parsed grammatical dependency tree to encode a bidirectional directed acyclic graph, as follows:

[0011] 2.1: Use the existing external grammar parser spaCy to process the input sentence and output the grammatical dependency graph of the sentence.

[0012] 2.2: Encoding according to the grammatical dependency tree structure

[0013] The adjacency matrix is constructed from the root node of the dependency tree to each leaf node, and a top-down directed acyclic graph structure is obtained; secondly, the sequence from each leaf node to the root node is encoded to obtain a bottom-up directed acyclic graph structure. Figure 1 Together they form a bidirectional directed acyclic graph, as follows:

[0014] 2.2.1: Construct a top-down directed acyclic graph to model the process of information flow from the root node of the syntax tree to the leaf nodes. All child nodes in the tree point to their parent nodes, and the root node does not point to other nodes, thereby simulating the low-level nodes receiving high-level node information.

[0015] 2.2.2: Construct a bottom-up directed acyclic graph to model the process of information flow from each leaf node of the syntax tree to the root node. All parent nodes in the tree point to their leaf nodes, and all nodes do not point to other nodes, thereby simulating high-level nodes receiving information from low-level nodes.

[0016] 3) Integrate word sentiment knowledge to obtain a bidirectional directed acyclic graph enhanced with sentiment knowledge.

[0017] 4) Use a bidirectional directed acyclic graph neural network to encode text representations with grammatical dependencies, as follows:

[0018] 4.1: Multiply the text representation with the bidirectional directed acyclic graph with sentiment and assign the corresponding text embedding to the corresponding text.

[0019] 4.2: Build a neural network capable of encoding a directed acyclic graph as follows:

[0020] 4.2.1: Utilize the Graph Attention Network (GAT) to aggregate the information of directly connected nodes for each node in the current layer to obtain the global sentiment information of the node in this layer.

[0021] 4.2.2: Node information unit: The gated recurrent unit (GRU) structure is used to allow the global emotional information of this layer to control the emotional expression of the node in the upper layer to propagate the node information of this layer.

[0022] 4.2.3: Text information unit, using the gated recurrent unit (GRU) structure to allow the emotional expression of the node in the upper layer to control the propagation of the global connection information of this layer to the text information of this layer.

[0023] 4.2.4: Mixed expression: add the node information expression of this layer and the text information propagation of this layer to obtain the mixed expression of the node as the emotional expression of the node in the current layer.

[0024] 4.3: Use directed acyclic graph neural networks to encode top-down and bottom-up directed acyclic graphs, respectively, to transmit sentiment information from top to bottom and bottom to top.

[0025] 5) Utilize the aspect-level masking mechanism to mask out the sentiment representation of non-aspect words.

[0026] 6) Utilize the attention mechanism with the previous text expression to fuse the masked high-order sentiment representation to obtain the high-order aspect sentiment representation.

[0027] 7) Apply linear transformation to obtain the final sentiment classification.

[0028] Compared with the prior art, the present invention has the following beneficial technical effects and significant technical progress:

[0029] 1) This paper proposes a bidirectional directed acyclic graph (DAG) to encode a grammatical dependency tree, simulating the bidirectional information transmission method in the tree and incorporating the sentiment knowledge of each word. This sentiment-enhanced BDAG structure and sequential node information transmission can extract high-level information even with a small number of stacked layers, which can handle sentiment analysis of complex and long sentences in the real world.

[0030] 2) The bidirectional directed acyclic graph neural network proposed in this invention does not produce obvious over-smoothing problems when stacking layers. By stacking network layers, the representation of each word can be continuously iteratively optimized to achieve more efficient emotion transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Flowchart of the present invention;

[0032] Figure 2 Schematic diagram of the specific operation of Example 1. DETAILED DESCRIPTION

[0033] In order to effectively extract high-order information and simulate the information flow in the grammatical dependency tree, the present invention proposes a bidirectional directed acyclic graph encoding method to simulate the bidirectional information flow in the grammatical dependency tree, and uses the emotional information from SenticNet to enhance the emotional expression ability of the bidirectional directed acyclic graph. Then, a directed acyclic graph neural network is designed to propagate the information of each node in the directed acyclic graph structure, and the top-down and bottom-up directed acyclic graphs are successively input into the directed acyclic graph neural network to transmit emotional information between words. Finally, the aspect mask is used to highlight the expression of aspect words, and the retrieval-based attention mechanism is used to further retrieve information from the context, input linear transformation, and classify the emotional polarity of the aspect words.

[0034] See attached Figure 1 The aspect-level sentiment classification method based on the directed acyclic graph network of the present invention specifically includes the following steps:

[0035] 1) Input the vectorized word matrix into the bidirectional long short-term memory network to extract text feature representation, as follows:

[0036] 1.1: Input the sentence to be processed into the BERT pre-trained model to obtain the word vector of each word, combine them into a word vector matrix, and extract the semantic information in the sentence.

[0037] 1.2: Input the word vector matrix into a bidirectional long short-term memory (LSTM) network for sequence modeling to obtain text feature representation.

[0038] 2) Use the parsed grammatical dependency tree to encode a bidirectional directed acyclic graph, as follows:

[0039] 2.1: Use the existing external grammar parser spaCy to process the input sentence and output the grammatical dependency graph of the sentence.

[0040] 2.2: Encode according to the grammatical dependency tree structure, build the adjacency matrix from the root node of the dependency tree to each leaf node, and obtain a top-down directed acyclic graph structure; then encode in the order from each leaf node to the root node, and obtain a bottom-up directed acyclic graph structure. Figure 1 Together they form a bidirectional directed acyclic graph, as follows:

[0041] 2.2.1: Construct a top-down directed acyclic graph to model the process of information flow from the root node of the syntax tree to the leaf nodes. All child nodes in the tree point to their parent nodes, and the root node does not point to other nodes, thereby simulating the low-level nodes receiving high-level node information.

[0042] 2.2.2: Construct a bottom-up directed acyclic graph to model the process of information flow from each leaf node of the syntax tree to the root node: all parent nodes in the tree point to their leaf nodes, and all nodes do not point to other nodes, thereby simulating high-level nodes receiving information from low-level nodes.

[0043] 3) Integrate word sentiment knowledge to obtain a bidirectional directed acyclic graph enhanced with sentiment knowledge.

[0044] 4) Using a bidirectional directed acyclic graph neural network to encode text representations with grammatical dependencies is as follows:

[0045] 4.1: Multiply the text representation with the bidirectional directed acyclic graph with sentiment and assign the corresponding text embedding to the corresponding text.

[0046] 4.2: Build a neural network capable of encoding a directed acyclic graph as follows:

[0047] 4.2.1: Utilize the Graph Attention Network (GAT) to aggregate the information of directly connected nodes for each node in the current layer to obtain the global sentiment information of the node in this layer.

[0048] 4.2.2: Node information unit, using the gated recurrent unit (GRU) structure to allow the global emotional information of this layer to control the emotional expression of the node in the upper layer to propagate the node information of this layer.

[0049] 4.2.3: Text information unit, using the gated recurrent unit (GRU) structure to allow the emotional expression of the node in the upper layer to control the propagation of the global connection information of this layer to the text information of this layer.

[0050] 4.2.4: Mixed expression: add the node information expression of this layer and the text information propagation of this layer to obtain the mixed expression of the node as the emotional expression of the node in the current layer.

[0051] 4.3: Use directed acyclic graph neural networks to encode top-down and bottom-up directed acyclic graphs, respectively, to transmit sentiment information from top to bottom and bottom to top.

[0052] 5) Utilize the aspect-level masking mechanism to mask out the sentiment representation of non-aspect words.

[0053] 6) Utilize the attention mechanism with the previous text expression to fuse the masked high-order sentiment representation to obtain the high-order aspect sentiment representation.

[0054] 7) Apply linear transformation to obtain the final sentiment classification.

[0055] The present invention will be further described below with reference to specific examples and accompanying drawings.

[0056] Example 1

[0057] See Figure 2 In this embodiment, the sentence to be processed is input into the BERT pre-trained model for vectorization, and then a bidirectional long short-term memory network is used for sequence modeling to obtain a text expression. At the same time, a grammatical dependency tree is obtained by parsing with a syntax tree parser and encoded into a bidirectional directed acyclic graph structure diagram; the word sentiment information in SenticNet is used to enhance the sentiment expression of the bidirectional directed acyclic graph; the text expression and the dependency graph are combined and encoded using a bidirectional directed acyclic graph neural network to propagate the sentiment information between words; the aspect-level masking mechanism is used to mask the expression of non-aspect words and highlight the sentiment expression of aspect words; then, the text expression generated by the previous bidirectional long short-term memory network is used to obtain a high-order sentiment expression using an attention mechanism; finally, a linear transformation is used to obtain the final sentiment classification.

[0058] The above description is only a preferred embodiment of the present invention, and certain modifications may be made thereto within the scope defined by the claims of the present invention, but all modifications will fall within the scope of protection of the present invention.

Claims

1. A method for aspect-level sentiment classification based on bidirectional directed acyclic graph, characterized by: The method comprises the following steps: 1) Input the vectorized word matrix into the bidirectional long short-term memory network to extract text feature representation; 2) Using the parsed grammatical dependency tree to encode a bidirectional directed acyclic graph; 3) Integrate word sentiment knowledge to obtain a bidirectional directed acyclic graph enhanced with sentiment knowledge; 4) Encoding text representations with grammatical dependencies using bidirectional directed acyclic graph neural networks; 5) Using aspect-level masking mechanism to mask out the sentiment representation of non-aspect words; 6) Utilizing the attention mechanism with the previous textual representation, the masked high-level sentiment representation is fused to obtain a high-level aspect sentiment representation; 7) Apply linear transformation to obtain the final sentiment classification; The step 4) specifically includes: 4.1: Multiply the text representation with the bidirectional directed acyclic graph with sentiment and assign the corresponding text embedding to the corresponding text; 4.2: Build a neural network capable of encoding directed acyclic graphs; 4.3: Use DAG neural networks to encode top-down and bottom-up DAGs and perform top-down and bottom-up information flows respectively.

2. The aspect-level sentiment classification method based on a bidirectional directed acyclic graph according to claim 1 is characterized in that: The step 1) specifically includes: 1.1: Input the sentence to be processed into the BERT pre-trained model to obtain the word vector of each word, combine it into a word vector matrix, and extract the semantic information in the sentence; 1.2: Input the word vector matrix into a bidirectional long short-term memory network for sequence modeling to obtain text feature representation.

3. The aspect-level sentiment classification method based on bidirectional directed acyclic graph according to claim 1 is characterized in that: The step 2) specifically includes: 2.1: Use the existing external parser spaCy to process the input sentence and output the grammatical dependency graph of the sentence; 2.2: To encode the grammatical dependency tree structure, first construct the adjacency matrix from the root node down to each leaf node to obtain a top-down directed acyclic graph, and then encode it in the order from each leaf node up to the root node to obtain a bottom-up directed acyclic graph structure. The two directed acyclic graphs together constitute a bidirectional directed acyclic graph.

4. The aspect-level sentiment classification method based on a bidirectional directed acyclic graph according to claim 3 is characterized in that: The step 2.2 specifically includes: 2.2.1: Construct a top-down directed acyclic graph to model the information flow from the root node of the syntax tree to the leaf nodes. All parent nodes in the tree point to their leaf nodes, and all leaf nodes do not point to other nodes, thus simulating the low-level nodes receiving information from high-level nodes. 2.2.2: Construct a bottom-up directed acyclic graph to model the information flow from the leaf nodes of the syntax tree to the root node. All child nodes in the tree point to their parent nodes, and the root node does not point to other nodes, thereby simulating high-level nodes receiving information from low-level nodes.

5. The aspect-level sentiment classification method based on a bidirectional directed acyclic graph according to claim 4 is characterized in that: The step 4.2 specifically includes: 4.2.1: Utilize the graph attention network to aggregate the information of directly connected nodes for each node in the current layer to obtain the global sentiment information of the node in this layer; 4.2.2: Using the gated recurrent unit structure, the global emotional information of this layer controls the emotional expression of the node in the upper layer to propagate the node information of this layer; 4.2.3: Using the gated recurrent unit structure, the emotional expression of the node in the upper layer controls the propagation of the global connection information of the current layer to the text information of the current layer; 4.2.4: Add the node information expression of this layer and the text information propagation of this layer to obtain the mixed expression of the node as the emotional expression of the node in the current layer.

Images