Aspect-level sentiment quadruple extraction method and system based on tree structure information perception

Abstract

This invention provides a method and system for aspect-level sentiment quadruples extraction based on tree structure information perception, comprising: acquiring a comment statement to be extracted, and constructing the comment statement into a sentiment analysis tree according to a predefined tree structure template; wherein the sentiment analysis tree is composed of word embedding vectors constituting the comment statement and a tree structure embedding representation; for the obtained sentiment analysis tree, using a text encoder to obtain the embedding representation of the sentiment analysis tree; for nodes in the sentiment analysis tree, performing aggregation representation on the information interaction between the current node and its neighboring nodes based on a graph attention network, and injecting the representation result into the embedding representation of the sentiment analysis tree to obtain a new embedding representation; based on the new embedding representation and a preset loss function, decoding to obtain all sentiment elements in the comment statement to be extracted, wherein the sentiment elements include aspect categories, aspect words, opinion words, and aspect sentiments.

Description

Technical Field

[0001] This invention belongs to the field of sentiment analysis technology, and in particular relates to a method and system for extracting aspect-level sentiment quadruples based on tree structure information perception. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Cai et al. studied the ASQP (Aspect Sentiment Quad Prediction) task, emphasizing implicit aspects or opinions. The authors argued that implicit aspects or opinions frequently appear in real-world scenarios and used "null" to represent their presence in the sentiment quad. They introduced two new datasets with sentiment quad annotations and built a series of pipeline baselines by combining existing models to benchmark the task. Later, researchers attempted to use the desired sentiment element sequence as the target of the generative model to solve the ABSA problem using a Seq2Seq approach; Zhang et al. proposed a paraphrase modeling strategy for end-to-end sentiment quad prediction. By combining annotated sentiment elements with pre-built templates and using the obtained natural language sentences as the target sequence, they transformed the original quad prediction task into a text generation problem, which was then solved using a Seq2Seq modeling paradigm. Subsequently, Hu et al. found that the order of sentiment elements in the templates affected the performance of quad extraction; they also proposed that combining multiple templates could improve the ASQP task through data augmentation. However, while these models can leverage rich label semantics by encoding natural language labels into the target output, they cannot effectively capture the semantic structure between aspect words and opinion words. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a method and system for aspect-level sentiment quadruples extraction based on tree structure information perception. The scheme models comment statements as a predefined sentiment analysis tree, then models each tag node in the tree into a prompt template of a pre-trained language model, effectively utilizing the structural information of the sentiment analysis tree. Simultaneously, a graph attention network is employed to learn the correlations between tag nodes and other nodes, injecting tree structure information into the template embedding, thereby effectively utilizing the connectivity between tags. This significantly improves the extraction accuracy of aspect-level sentiment quadruples, laying the foundation for downstream tasks.

[0005] According to a first aspect of the present invention, a method for extracting aspect-level sentiment quadruples based on tree structure information perception is provided, comprising:

[0006] The process involves obtaining comment statements to be extracted and constructing them into a sentiment analysis tree according to a predefined tree structure template. The sentiment analysis tree is composed of word embedding vectors that constitute the comment statements and tree structure embedding representations.

[0007] For the obtained sentiment analysis tree, the text encoder is used to obtain the embedded representation of the sentiment analysis tree;

[0008] For nodes in the sentiment analysis tree, a graph attention network is used to aggregate the information interaction between the current node and its neighboring nodes, and the representation result is injected into the embedding representation of the sentiment analysis tree to obtain a new embedding representation.

[0009] Based on the new embedding representation and the preset loss function, all sentiment elements in the comment statement to be extracted are obtained by decoding. The sentiment elements include aspect categories, aspect words, opinion words, and aspect sentiments.

[0010] Furthermore, the graph attention network aggregates and represents the information interactions between the current node and its neighboring nodes, specifically as follows:

[0011]

[0012] Where v represents a node in the set of neighboring nodes of node u. This represents the graph attention representation of node v at level k, where k denotes the k-th level of the graph attention network, N(u) represents the neighboring nodes of node u, and c u It is a normalization constant. These are trainable parameters.

[0013] Furthermore, in the template of the sentiment analysis tree, each template word is followed by a preset marker for predicting tree nodes.

[0014] Furthermore, by using preset markers to divide predictions at different locations, the system learns the dependencies between different labels.

[0015] Furthermore, in order to effectively utilize tree node knowledge, several virtual nodes are created and connected to the corresponding nodes in the sentiment analysis tree embedding representation to obtain a new embedding representation of the nodes in the sentiment analysis tree embedding representation.

[0016] Furthermore, the text encoder employs the BERT model.

[0017] Furthermore, the preset loss function includes a masked language model loss and a binary cross-entropy loss function, wherein the binary cross-entropy loss function is set separately based on aspect category, aspect word, opinion word, and aspect sentiment.

[0018] According to a second aspect of the present invention, an aspect-level sentiment quadruple extraction system based on tree structure information perception is provided, comprising:

[0019] A sentiment analysis tree construction unit is used to obtain the comment statements to be extracted and construct the comment statements into a sentiment analysis tree according to a predefined tree structure template; wherein, the sentiment analysis tree is composed of word embedding vectors constituting the comment statements and tree structure embedding representation;

[0020] An embedding representation unit is used to obtain an embedding representation of the obtained sentiment analysis tree using a text encoder.

[0021] The tree structure information injection unit is used to aggregate the information interaction between the current node and its neighboring nodes based on the graph attention network for nodes in the sentiment analysis tree, and inject the representation result into the embedding representation of the sentiment analysis tree to obtain a new embedding representation.

[0022] The quadruple extraction unit is used to obtain all sentiment elements of the comment statement to be extracted by decoding based on the new embedding representation and the preset loss function. The sentiment elements include aspect categories, aspect words, opinion words and aspect sentiments.

[0023] According to a third aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the memory, wherein the processor executes the program to implement the aforementioned method for extracting aspect-level sentiment quadruples based on tree structure information perception.

[0024] According to a fourth aspect of the present invention, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the aforementioned method for extracting aspect-level sentiment quadruples based on tree-structured information perception.

[0025] The above one or more technical solutions have the following beneficial effects:

[0026] (1) This invention provides a method and system for extracting aspect-level sentiment quadruplets based on tree structure information perception. The scheme models the comment statement as a predefined sentiment analysis tree, and then models each tag node in the tree into the prompt template of the pre-trained language model, so as to effectively utilize the structural information of the sentiment analysis tree. At the same time, a graph attention network is used to learn the correlation between tag nodes and other nodes, and the tree structure information is injected into the template embedding to realize the effective utilization of the connectivity between tags, thereby effectively improving the extraction accuracy of aspect-level sentiment quadruplets and laying the foundation for the implementation of downstream tasks.

[0027] (2) To better utilize tree node knowledge, the proposed scheme creates several virtual nodes and connects them to corresponding nodes in the sentiment analysis tree embedding representation. In this way, these virtual nodes can aggregate information from a specific node through manual connections, further improving the extraction accuracy of aspect-level sentiment quadruples.

[0028] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0029] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0030] Figure 1 This is a flowchart of an aspect-level sentiment quadruple extraction method based on tree structure information perception, as described in an embodiment of the present invention.

[0031] Figure 2 This is a schematic diagram of the tree structure described in an embodiment of the present invention. Detailed Implementation

[0032] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0033] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0034] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0035] Example 1:

[0036] The purpose of this embodiment is to provide a method for extracting aspect-level sentiment quadruples based on tree structure information perception.

[0037] A method for extracting aspect-level sentiment quadruples based on tree-structured information perception includes:

[0038] The process involves obtaining comment statements to be extracted and constructing them into a sentiment analysis tree according to a predefined tree structure template. The sentiment analysis tree is composed of word embedding vectors that constitute the comment statements and tree structure embedding representations.

[0039] For the obtained sentiment analysis tree, the text encoder is used to obtain the embedded representation of the sentiment analysis tree;

[0040] For nodes in the sentiment analysis tree, a graph attention network is used to aggregate the information interaction between the current node and its neighboring nodes, and the representation result is injected into the embedding representation of the sentiment analysis tree to obtain a new embedding representation.

[0041] Based on the new embedding representation and the preset loss function, all sentiment elements in the comment statement to be extracted are obtained by decoding. The sentiment elements include aspect categories, aspect words, opinion words, and aspect sentiments.

[0042] In specific implementation, the graph attention network aggregates and represents the information interaction between the current node and its neighboring nodes, specifically as follows:

[0043]

[0044] Where v represents a node in the set of neighboring nodes of node u. This represents the graph attention representation of node v at level k, where k denotes the k-th level of the graph attention network, N(u) represents the neighboring nodes of node u, and c u It is a normalization constant. These are trainable parameters.

[0045] In specific implementation, each template word in the sentiment analysis tree template is followed by a preset label for predicting tree nodes; at the same time, the predictions at different positions are divided by the preset labels to learn the dependency relationship between different labels.

[0046] In practical implementation, in order to effectively utilize tree node knowledge, several virtual nodes are created and connected to the corresponding nodes in the sentiment analysis tree embedding representation to obtain a new embedding representation of the nodes in the sentiment analysis tree embedding representation.

[0047] In practice, the text encoder uses the BERT model.

[0048] In specific implementation, the preset loss function includes masked language model loss and binary cross-entropy loss function, wherein the binary cross-entropy loss function is set separately based on aspect category, aspect word, opinion word and aspect sentiment.

[0049] For ease of understanding, the solution described in this embodiment will be explained in detail below with reference to the accompanying drawings:

[0050] In aspect-level sentiment analysis, the sentiment elements are not independent or orthogonal; hierarchical dependencies exist between them. Through experiments on a dataset, we used a probabilistic method to predict the corresponding aspect words and sentiment polarities using only aspect categories and opinion words, respectively. The experimental results validated our hypothesis. Specifically, we found that when using only aspect categories, aspect words could be accurately extracted from comments with a 61.60% probability; similarly, when using only opinion words, the corresponding aspect sentiment polarity could be obtained with a 77.55% probability. These results indicate a dependency between aspect categories and aspect words, as well as between opinion words and sentiment polarity. Therefore, the traditional approach of treating aspect sentiment element extraction as a sequence-to-sequence task is inadequate.

[0051] This embodiment provides an aspect-level sentiment quadruple extraction method based on tree structure information awareness. Since there are dependencies between aspect sentiment elements, treating aspect sentiment element extraction as a sequence-to-sequence task is not rigorous. Therefore, we first model the comment statement as a predefined sentiment analysis tree, and then model each label node in the tree into the prompt template of a pre-trained language model. This allows for better utilization of the structural information of the sentiment analysis tree, but the sentiment elements lack connectivity. To fully utilize the tree structure in a masked language model (MLM) manner, we further inject tree structure knowledge into the template embedding. That is, we use a graph attention network (GAT) to learn the correlation between label nodes and other nodes. Finally, after reconstruction by the pre-trained language model, we can decode all the sentiment elements we need. We conducted a complete analysis of the model and its different components, and evaluated the proposed model on two commonly used datasets. Our model outperforms the comparison methods.

[0052] like Figure 1 As shown, the solution described in this embodiment specifically includes the following steps:

[0053] (1) Problem Definition

[0054] Given a sentence, S = {x1, x2, ..., x...} n The goal is to obtain all the sentiment quadruples contained in the sentence, namely {C1,A1,O1,P1},…,{C n A n O n ,P n}, where C represents aspect category, A represents aspect word, O represents opinion word, and P represents aspect emotion.

[0055] It's important to note that a sentence typically contains multiple aspects and opinion words. The quadruple extraction task not only identifies the four elements but also combines them into a valid quadruple, while considering implicit aspects / opinions. Since implicit aspects / opinions cannot be explicitly expressed as a word or phrase, we leave A empty in the case of implicit aspects and use category C to describe opinion words; in the case of implicit opinions, we leave O empty and use sentiment S to describe its semantic orientation.

[0056] (2) Tree structure constraints

[0057] To preserve more information about the tree structure, we propose a prompt-learning fine-tuning method based on tree structure reconstruction. Since the label structure is tree-like, we construct the prompt based on the tree structure. For simplicity, we define the tree structure as follows: Figure 2 As shown ( Figure 2 The comment statement shown contains two sentiment quadruples. Given input text x and a predefined aspect sentiment tree with the template [CLS]x[SEP][tc1][PRED][t p1 ][PRED][t a1 ][PRED][t o1 [PRED]...[t cM ][PRED][y pM ][PRED][t aM ][PRED][t oM [PRED]. We use a special label [PRED] for label prediction. Where M is 4*S, and S is the number of quadruplets in the comment statement.

[0058] Among them, t in the template C1 This refers to the label of the first aspect category, followed by [PRED] which indicates the prediction of the label, and so on. P1 This refers to the first sentiment polarity label, t A1 This refers to the first aspect tag, t o1 This refers to the first opinion tag, t. oM This refers to the Mth opinion word tag.

[0059] Setting the opinion word O to empty, as mentioned above, is only effective when the opinion word is implicit, as explicitly stated above. Implicit aspect or opinion refers to a comment statement that does not explicitly indicate the aspect or opinion word, such as the comment statement "Had a party here." In this statement, neither the aspect nor the opinion word is explicitly stated; only the aspect category is "restaurant," and the sentiment polarity is positive. Therefore, in this case, we set A and O to empty.

[0060] (3) Encoding module

[0061] We use BERT as the text encoder, which first embeds tokens into the input. Taking the comment statement in the model graph as an example:

[0062] T = [x1, x2, ..., x N ,t c1 ,e P ,t p1 ,e P ,…,t o2 ,e P ]

[0063] Where X = [x1, x2, ..., x N ] is the word embedding vector of the input text, e P The embedding vector for the special prediction label [PRED] is initialized with the BERT [MASK] tag, [t c1 ,e P ,t p1 ,e P ,…,t o2 ,e P [] represents a tree-structured prompt embedding. The prompt embedding is randomly initialized and learns relevant knowledge through training. To simplify the explanation of the principle, we have omitted the special tags [CLS] and [SEP] here.

[0064] Then T is encoded to obtain its hidden representation:

[0065]

[0066] in It is the i-th e P Hidden representation of nodes.

[0067] For tag word prediction, we predict each tag y i Create a learnable virtual tag word v i It initializes its embedding v with the average embedding of its corresponding tokens. i Instead of predicting all labels in a single slot, we group the labels into different groups based on the tree's label structure and constrain [PRED] to predict only the label on a single node. For this purpose, each template word [t]... iEach label is followed by a [PRED] marker, used for the prediction of the i-th tree node. By dividing the prediction into different slots, the model can better learn the dependencies between different labels. Here, the label corresponds to the aspect category, aspect word, opinion word, and aspect sentiment to be predicted; the slot refers to the slot for predicting a label, such as... Figure 1 As shown, in the BCE Loss section, we predict each label individually, rather than predicting all labels in one slot.

[0068] Therefore, for We define its tag prediction V m for:

[0069]

[0070] Where N is the set of tags for emotional elements, Indicates other.

[0071] (4) Tree structure information injection

[0072] Tree structure constraints only describe the relationships between tags, but lack their connectivity. To fully utilize the tree structure in an MLM-like manner, we further inject tree structure knowledge into template embedding.

[0073] We employ a K-layer stacked Graph Attention Network (GAT) to simulate the label structure. Given a node u in the k-th layer of the GAT, the information interaction and aggregation operations are defined as follows:

[0074]

[0075] Where N(u) represents the neighboring nodes of node u, c u It is a normalization constant. These are trainable parameters.

[0076] To better utilize the knowledge of tree nodes, we created M virtual nodes t1, t2, ..., t M (M is 4*S, where S is the number of quadruplets), then the virtual node t i Connect it to the i-th node in H. In this way, these virtual nodes can aggregate information from a specific node through artificial connections. For the first GAT layer, we use node y... i The virtual label word vi ∈Y is used as its node feature, and the prompt embedding is assigned to the virtual node t. i As its node characteristics.

[0077] Then GAT is applied to the new graph, and the virtual node t is output. i The representation of This node has already collected knowledge from the i-th node. We utilize residual connections to implement the template embedding of the i-th graph:

[0078]

[0079] The new t' embeds tree structure information i Injected into BERT to replace the t in the input text i .

[0080] (5) Loss Function

[0081] Our method's loss function consists of two parts: MLM loss and binary cross-entropy loss.

[0082] (6) MLM loss

[0083] We randomly mask 15% of the words in the text to calculate the MLM loss. M .

[0084] (7) Binary cross-entropy loss

[0085] We use binary cross-entropy loss as the loss function for classification. For each fine-grained sentiment element (four types: aspect category, aspect sentiment polarity, aspect item, and opinion item), we use a different classifier head. We classify each of these elements as follows: PRED The binary cross-entropy loss is calculated, and if an aspect or opinion term has more than one word, their average value is used. The losses for the four classification heads are as follows:

[0086]

[0087]

[0088]

[0089]

[0090] (8) Joint training

[0091] Finally, the overall loss function can be expressed as:

[0092] L=λ1L M LM+λ2L C +λ3L P +λ4L A +λ5L O

[0093] Where λ is a hyperparameter used to determine the contribution of each task.

[0094] Example 2

[0095] The purpose of this embodiment is to provide an aspect-level sentiment quadruple extraction system based on tree structure information perception.

[0096] A tree-structured information perception-based aspect-level sentiment quadruple extraction system includes:

[0097] A sentiment analysis tree construction unit is used to obtain the comment statements to be extracted and construct the comment statements into a sentiment analysis tree according to a predefined tree structure template; wherein, the sentiment analysis tree is composed of word embedding vectors constituting the comment statements and tree structure embedding representation;

[0098] An embedding representation unit is used to obtain an embedding representation of the obtained sentiment analysis tree using a text encoder.

[0099] The tree structure information injection unit is used to aggregate the information interaction between the current node and its neighboring nodes based on the graph attention network for nodes in the sentiment analysis tree, and inject the representation result into the embedding representation of the sentiment analysis tree to obtain a new embedding representation.

[0100] The quadruple extraction unit is used to obtain all sentiment elements of the comment statement to be extracted by decoding based on the new embedding representation and the preset loss function. The sentiment elements include aspect categories, aspect words, opinion words and aspect sentiments.

[0101] Furthermore, the system described in this embodiment corresponds to the method described in Embodiment 1, and its technical details have been described in detail in Embodiment 1, so they will not be repeated here.

[0102] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0103] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for extracting aspect-level sentiment quadruples based on tree structure information perception, characterized in that, include: The process involves obtaining comment statements to be extracted and constructing them into a sentiment analysis tree according to a predefined tree structure template. The sentiment analysis tree is composed of word embedding vectors that constitute the comment statements and tree structure embedding representations. For the obtained sentiment analysis tree, the text encoder is used to obtain the embedded representation of the sentiment analysis tree; For nodes in the sentiment analysis tree, a graph attention network is used to aggregate and represent the information interactions between the current node and its neighboring nodes. The graph attention-based network aggregates and represents the information interactions between the current node and its neighboring nodes, specifically as follows: Where v represents a node in the set of neighboring nodes of node u. This represents the graph attention representation of node v at layer k, where k denotes the k-th layer of the graph attention network. Indicates the neighboring nodes of node u. It is a normalization constant. These are trainable parameters, and the representation results are injected into the embedding representation of the sentiment analysis tree to obtain a new embedding representation; Based on the new embedding representation and the preset loss function, all sentiment elements of the comment statement to be extracted are obtained by decoding. The sentiment elements include aspect categories, aspect words, opinion words and aspect sentiments. To effectively utilize tree node knowledge, several virtual nodes are created and connected to the corresponding nodes in the sentiment analysis tree embedding representation to obtain a new embedding representation of the nodes in the sentiment analysis tree embedding representation.

2. The aspect-level sentiment quadruple extraction method based on tree structure information perception as described in claim 1, characterized in that, In the template of the sentiment analysis tree, each template word is followed by a preset marker for predicting tree nodes.

3. The aspect-level sentiment quadruple extraction method based on tree structure information perception as described in claim 2, characterized in that, By dividing the predictions at different positions using preset tags, the system learns the dependencies between different labels.

4. The aspect-level sentiment quadruple extraction method based on tree structure information perception as described in claim 1, characterized in that, The text encoder uses the BERT model.

5. The aspect-level sentiment quadruple extraction method based on tree structure information perception as described in claim 1, characterized in that, The preset loss function includes a masked language model loss and a binary cross-entropy loss function, wherein the binary cross-entropy loss function is set separately based on aspect category, aspect word, opinion word, and aspect sentiment.

6. A system for extracting aspect-level emotion quadruples based on tree-structured information perception, characterized in that, include: A sentiment analysis tree construction unit is used to obtain the comment statements to be extracted and construct the comment statements into a sentiment analysis tree according to a predefined tree structure template; wherein, the sentiment analysis tree is composed of word embedding vectors constituting the comment statements and tree structure embedding representation; An embedding representation unit is used to obtain an embedding representation of the obtained sentiment analysis tree using a text encoder. The tree structure information injection unit is used to aggregate and represent the information interaction between the current node and its neighboring nodes based on a graph attention network for nodes in the sentiment analysis tree. The graph attention-based network aggregates and represents the information interactions between the current node and its neighboring nodes, specifically as follows: Where v represents a node in the set of neighboring nodes of node u. This represents the graph attention representation of node v at layer k, where k denotes the k-th layer of the graph attention network. Indicates the neighboring nodes of node u. It is a normalization constant. These are trainable parameters, and the representation results are injected into the embedding representation of the sentiment analysis tree to obtain a new embedding representation; The quadruple extraction unit is used to obtain all sentiment elements of the comment statement to be extracted by decoding based on the new embedding representation and the preset loss function. The sentiment elements include aspect categories, aspect words, opinion words and aspect sentiments. To effectively utilize tree node knowledge, several virtual nodes are created and connected to the corresponding nodes in the sentiment analysis tree embedding representation to obtain a new embedding representation of the nodes in the sentiment analysis tree embedding representation.

7. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and running thereon, characterized in that, When the processor executes the program, it implements an aspect-level sentiment quadruple extraction method based on tree structure information perception as described in any one of claims 1-5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements an aspect-level sentiment quadruple extraction method based on tree structure information perception as described in any one of claims 1-5.

Images