Aspect-level sentiment analysis method based on knowledge assistance and sentiment relation driving

Abstract

The invention provides an aspect-level sentiment analysis method based on knowledge assistance and sentiment relation driving, and relates to the technical field of sentiment analysis, which comprises the steps of performing multi-task definition, constructing input representation, searching and weighting a common sense path based on an external knowledge base, constructing a common sense perception graph and updating node representation, and performing multi-task joint training. And finally realizing emotion prediction. According to the method, the sentiment relation between sentence sentiment fundamental tones and aspect words is introduced, the sentiment-emphasized perception ability of the model for commentators is enhanced, external common knowledge is fused, the semantic understanding and implicit relation reasoning ability is improved, and a multi-task learning framework is adopted; the method assists tasks in improving robustness and interpretability of aspect-level sentiment analysis through sentiment relation recognition, is suitable for actual scenes such as user comment analysis and product feedback mining, and helps enterprises and institutions deeply understand user sentiment tendency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sentiment analysis, in particular to an aspect-level sentiment analysis method based on knowledge assistance and sentiment relationship driving. BACKGROUND

[0002] Most of the previous scholars' research focuses on modeling the relationship between aspect words and context. Although these methods are effective, they only focus on the interaction between aspect words and context, ignoring the emotional tone of the sentence itself. The purpose of the reviewer's review is to express satisfaction and subsequent improvement expectations, which has emotional emphasis characteristics. Emotional emphasis helps highlight the reviewer's personal preferences, focus and importance of different emotional elements in mixed emotional expression. It provides more rich emotional information, enabling readers to better understand the reviewer's emotional experience and attitude. The emotional emphasis tone of the review sentence itself, as an important emotional element of emotional expression, can better understand the reviewer's emotional experience and tendency by perceiving the relationship between specific aspect words and the emotional tone of the review sentence itself. However, how to utilize aspect word emotion and review sentence emotional emphasis becomes a challenge.

[0003] In addition, in order to ensure accurate inference of emotional relationships, the model must deeply understand the semantic information implied in aspects and sentences. In previous studies, relying only on limited coarse-grained emotional cues in the context is not enough to fully realize semantic understanding and implicit relationship reasoning. External knowledge can provide rich background information and domain knowledge, which can enhance the understanding and reasoning ability of the model. Therefore, in order to better perform text semantic parsing and implicit relationship reasoning, the assistance of external common sense knowledge is needed.

[0004] Therefore, it is very necessary to design an aspect-level sentiment analysis method based on knowledge assistance and sentiment relationship driving. SUMMARY

[0005] In order to overcome the shortcomings of the prior art, the purpose of the present application is to provide an aspect-level sentiment analysis method based on knowledge assistance and sentiment relationship driving.

[0006] To achieve the above purpose, the present application provides the following scheme: The present application provides an aspect-level sentiment analysis method based on knowledge assistance and sentiment relationship driving, comprising the following steps: Step 1: multi-task definition is performed, and input representation is constructed; Step 2: find and weight the common sense path based on the external knowledge base; Step 3: build a common sense perception graph and update the node representation; Step 4: finally realize sentiment prediction by multi-task joint training.

[0007] Preferably, in step 1, multi-task definition is performed, specifically: Two sub-tasks are defined, respectively: Aspect-level sentiment classification task: given a sentence and an aspect word sequence , determine the sentiment polarity of each aspect word, wherein the sentiment polarity includes positive, neutral and negative; Sentiment relation identification task: define the relationship between aspect word sentiment and sentence sentiment tone, which includes: When the aspect word sentiment and the sentence sentiment are the same, the relationship is defined as similar; When the sentiments are opposite, the relationship is defined as opposite; When only one of the sentiments is neutral, the relationship is defined as irrelevant. When only one of the sentiments is neutral, it is specifically: when the sentence and the aspect word sentiment are neutral, as long as one of the two is neutral.

[0008] Preferably, in step 1, the input representation is constructed, specifically: Based on a pre-trained language model As an encoder, construct a sentence-aspect word pair as input, generate a sentence representation vector with aspect awareness as input representation.

[0009] Preferably, in step 2, the common sense path is found and weighted based on the external knowledge base, specifically: Use SenticNet sentiment dictionary to identify sentiment words in the sentence; Take the aspect word as the head node and the identified sentiment word as the tail node, find the common sense path connecting them in the common sense knowledge graph ConceptNet, and keep at least three entity information paths for each aspect word-sentiment word pair; Calculate the similarity between the sentence representation vector and the path representation, and use it as the path weight to obtain the weighted path representation.

[0010] Preferably, in step 3, the common sense perception graph is constructed and the node representation is updated, specifically: Based on the sentence representation vector and the weighted path representation, construct a common sense perception graph; Update the nodes in the common sense perception graph based on the common sense path network of the relational graph attention network.

[0011] Preferably, the nodes in the common sense perception graph are updated based on the common sense path network of the relational graph attention network, specifically: Dependency node update: aggregate the information of each word and its neighbor nodes on the syntactic dependency edge to update the representation of the word to incorporate syntactic structure information; Common sense relationship node update: For nodes connected by common sense path edges, calculate the matching degree between aspect word vectors, path vectors and sentiment word vectors, and update the node representation accordingly; From the updated total node representations, extract the node representations corresponding to the aspect words, and generate the final aspect word representation vector through average pooling.

[0012] Preferably, in step 4, sentiment prediction is ultimately achieved through multi-task joint training, specifically as follows: The aspect word representation vector is input into a fully connected layer, the Softmax function is used to predict the sentiment polarity distribution of the aspect words, and the cross-entropy loss of the aspect-level sentiment classification task is calculated. The sentence representation vector and aspect word representation vector are concatenated and input into another fully connected layer to predict the relationship between aspect word sentiment and sentence sentiment tone, and to calculate the cross-entropy loss for the relationship recognition task. The losses of the two tasks are weighted and summed to obtain the total loss function of the model. The model is then trained end-to-end by minimizing the total loss. In the prediction phase, the trained model is used to output the sentiment polarity classification results of the aspect words.

[0013] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This invention provides an aspect-level sentiment analysis method based on knowledge assistance and emotion relationship driving. The method includes defining multiple tasks and constructing input representations, searching and weighting common sense paths based on an external knowledge base, constructing a common sense perception graph and updating node representations, and finally achieving sentiment prediction through multi-task joint training. This invention introduces the sentiment relationship between sentence sentiment tone and aspect words to enhance the model's ability to perceive the commenter's emotional emphasis, integrates external common sense knowledge to improve semantic understanding and implicit relationship reasoning capabilities, and adopts a multi-task learning framework. Through the emotion relationship identification auxiliary task, it improves the robustness and interpretability of aspect-level sentiment analysis, making it applicable to practical scenarios such as user comment analysis and product feedback mining, helping enterprises and institutions to deeply understand user sentiment tendencies. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the overall framework of the method of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] The purpose of this invention is to provide an aspect-level sentiment analysis method based on knowledge assistance and emotional relationship driving. It introduces the emotional relationship between sentence sentiment tone and aspect words to enhance the model's ability to perceive the emotional focus of commenters. It integrates external common sense knowledge to improve semantic understanding and implicit relationship reasoning capabilities. It adopts a multi-task learning framework and improves the robustness and interpretability of aspect-level sentiment analysis through the emotional relationship identification auxiliary task. It is applicable to practical scenarios such as user comment analysis and product feedback mining, helping enterprises and institutions to deeply understand users' emotional tendencies.

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] like Figure 1 As shown, this invention provides an aspect-level sentiment analysis method based on knowledge assistance and emotional relationship-driven approaches, comprising the following steps: Step 1: Define multiple tasks and construct the input representation; Step 2: Find and weight common sense paths based on external knowledge bases; Step 3: Construct a commonsense perception graph and update the node representations; Step 4: Through multi-task joint training, sentiment prediction is finally achieved.

[0020] In step 1, multi-task definition is performed, specifically as follows: Define two subtasks, namely: Aspect-level sentiment classification task: Given a... Context words and Sentences with various aspects Representing sentences and aspect word sequences as and In this task, the aspect word sequence is a subset of the comment sentence sequence, and the goal is to determine the sentiment polarity (positive, neutral, and negative) of each aspect word in S. Sentiment Relationship Recognition Task: Define the relationship between aspect word sentiment and sentence sentiment tone, including: When the aspect term sentiment and the sentence sentiment are the same, the relationship is defined as similar; When emotions are opposite, the relationship is defined as opposite; A relationship is defined as irrelevant when only one of its emotions is neutral.

[0021] In step 1, the input representation is constructed, specifically as follows: This invention employs a pre-trained language model. BERT serves as the encoder for the Information Aid and Sentiment-Driven Sentiment Analysis (IASD) model. It possesses excellent word embedding feature representation capabilities and has been widely applied in natural language processing tasks. First, it constructs a sentence-aspect word pair: As input to BERT, we obtain aspect-aware sentence representation vectors. For path information, the input sequence... The path is fed into BERT to obtain a representation of the path. Sentence represents a comment sentence, and aspect represents an aspect term. The characters are the input format for the BERT model, where [cls] represents the start character and [sep] represents the separator or end character.

[0022] Before feeding the "sentence-aspect word pair" into BERT, this embodiment performs normalization preprocessing: Unicode standardization (NFKC) is applied to the input text, full-width characters and punctuation are normalized, emoticons and special symbols are replaced with placeholders (e.g., [EMOJI]), and numeric masks are applied (e.g., long numeric strings are replaced with the unified notation [NUM]). WordPiece segmentation is then performed according to the BERT tokenizer. To explicitly mark the boundaries of aspect words, aspect position markers (custom nouns, AspectPosition Markers, APM) are introduced. Aspect word fragments are enclosed in the original sentence with [ASP] and [ / ASP], and then a sequence is constructed according to BERT's "sentence pair" input format: [CLS]sentence_Tok(withAPM)[SEP]aspect_Tok[SEP], where Segment-ID distinguishes between sentence segments and aspect segments. The above processing only affects the organization of the input sequence and does not change the BERT network structure or subsequent multi-task settings.

[0023] To enhance the aspect sensitivity of the sentence representation vector, this embodiment adds an Aspect-Distance Embedding (ADE) to each token at the BERT input as an additional trainable vector. Let the sentence segment be... The distance from each token (WordPiece granularity) to the center position of the aspect term is

[0024] in The Token index for the center of the aspect word segment, is the upper limit of the distance to achieve truncation. Then the final input vector of the th Token is

[0025] where is the word vector, is the absolute position vector, is the Segment vector, is the ADE vector corresponding to the distance bucket (obtained by trainable look-up table). ADE is only added in the input embedding layer without changing the internal self-attention structure of BERT; the parameters come from end-to-end learning of the training data, and the value range is the same as the BERT embedding dimension. Its function is to give higher discriminability to the local context close to the aspect word in the representation layer, so as to form a sentence representation vector with aspect awareness For the path passed in by step 2 (the candidate common sense path obtained by DFS), in this embodiment, only the linearization and annotation methods are specified in step 1 for BERT encoding: for the path (the nodes are entities and the edges are relationships), construct the sequence [CLS] [ENT]v_0 [REL] r_1 [ENT]v_1 … [REL] r_L [ENT]v_L [SEP], where [ENT] and [REL] are structure role markers (used to indicate the role of the subsequent Token sequence representing the entity or relationship type), and the direction information is reserved in the relationship name after [REL] (such as ”Causes→“ or ”一Causes”). The sequence is segmented and encoded by the same BERT tokenizer as the sentence end to obtain the hidden states at each position . To obtain a stable path representation , Nodes-OnlyPooling (a custom term) is adopted: only the average of the hidden states corresponding to the entity word segments immediately following [enT] is taken (or the [cLS] vector is taken as a degenerate case) to avoid the interference of relationship markers on the entity semantics To ensure the consistency of calculating weights (formulas (1)(2)) with ”sentence vector - path vector similarity” in step 2 later, in this embodiment, linear alignment and normalization are performed on these two types of representations in step 1: Let the sentence end representation obtained by BERT (taking [cis] or the average of the APM-inner segments) be , and the path end be . If their dimensions and scales are different, then

[0026] where is the trainable linear projection matrix (mapping the representation to a common dimension ), then apply Normalization is used so that cosine similarity can be directly used as the basis for weight calculation in subsequent step 2. The above alignment and normalization belong to the scale unification at the "input representation construction" level.

[0027] This invention provides a set of preferred values: maximum input length 128; maximum aspect term length 6 WordPieces; preprocessing placeholder set size 8 (e.g., [NUM], [DATE], [EMOJI], etc.); ADE distance upper limit D. max =16, distance bucket number 17; embedding and BERT hidden dimension d model =768; Node pooling uses simple averaging, with a minimum of 2 entity points; if insufficient, [CLS] is used; Linear alignment projection dimension is also 768; After alignment, a unified process is performed. Normalization; the default value for the sentence is [CLS]. If the sentence contains APM, the average of the segments between [ASP] and [ / ASP] is taken as an alternative and concatenated with [CLS] before being linearly projected to 768.

[0028] In step 2, common sense paths are searched and weighted based on external knowledge bases, specifically as follows: This invention uses SenticNet as a sentiment dictionary to match and find sentiment words in sentences. SenticNet is a sentiment information analysis system that can be used as a sentiment dictionary to parse sentiment words in sentences. Then, the aspect words are used as head nodes, and the sentiment words obtained by each SenticNet are used as tail nodes. The depth-first search algorithm is applied in ConceptNet to find the corresponding common sense information path. Although ConceptNet has rich knowledge path information, shorter paths are more likely to provide reliable reasoning evidence. Therefore, this invention retains only three information paths containing at least three entities for each entity pair. Furthermore, the three selected constant knowledge paths do not contribute to the commentary sentence in the same way. Therefore, different weights need to be assigned to different path information to ensure that the path information conforms to the implicit background information of the commentary sentence and plays a role in interpretability. Specifically, this invention utilizes the aspect-aware hidden vectors output by BERT. (i.e., sentence representation vector) serves as the query vector for each knowledge path. Consider it as a query object (P) t H represents each path. p Let J represent the set of J paths. The similarity between the sentence vector and the path vector is calculated, and the similarity is used as the path weight coefficient. The specific calculation process is as follows: (1) (2) In the formula, Sentence vector The transpose of , where softmax represents the activation function. These are the weight coefficients of the sentence vector and the path vector. It is the information path representation after attention is allocated.

[0029] In step 3, a commonsense awareness graph is constructed and the node representations are updated, specifically as follows: Construct a commonsense perception graph based on sentence representation vectors and weighted path representations; The commonsense path network based on relational graph attention network updates the nodes in the commonsense perception graph.

[0030] A commonsense-aware graph is constructed based on sentence representation vectors and weighted path representations, specifically as follows: First, a common sense perception graph G(V, E) is constructed, where node V represents a word node, and edge E is divided into word dependency edges D-Edges and common sense knowledge path edges C-Edges. When there is a grammatical dependency between two words, edge D-Edges is 1, otherwise it is 0. When there is a common sense knowledge path between two words, edge C-Edges is 1, otherwise it is 0. In addition, the knowledge information path is mapped into a vector to fully explore its implicit sentiment cues, thereby helping to measure whether aspect words and sentiment words match.

[0031] The commonsense path network based on the relational graph attention network updates the nodes in the commonsense perception graph, specifically as follows: Dependency node updates: Considering the grammatical relationships and common-sense path knowledge in the common-sense perception graph, the model uses a RGAT-based CPN to update word representations. The CPN constructed in this invention not only considers the dependencies between words and their dependency types, but also the sentiment background information and sentiment cues implied in the knowledge path information. Therefore, the CPN can generate knowledge-rich word representations; The core of CPN is to use the syntactic relation vectors between words as features to represent the relationships between words. Considering word dependency edges (D-Edges), CPN aggregates the dependency features into the calculation of the attention between words and their neighboring nodes through multi-head attention, thereby updating the word representation of each node as follows: (3) (4) (5) In the formula, Represents the number of layers in the CPN. The k-th attention head represents multi-head attention. It is the dimension of the word vector. It is a scaled dot product attention function. It is a ReLU activation function. Representative words and Word dependencies between them Representatives and words A set of syntactically adjacent nodes. , , , and These are trainable weight parameters.

[0032] Common sense relationship node update: Inspired by graph embedding theory (that a tail node is close to the result of adding a relation vector to the head node in a low-dimensional vector space), this invention assumes that an aspect-sentiment pair contains a possible path reasoning process described by C-edges. That is, in the current context, if there is a possible reasoning process described by C-edges between the aspect word and the connected node, then that node represents the aspect's sentiment. Therefore, this invention holds that: aspect word vector + common-sense knowledge path vector = sentiment word vector. Therefore, scaling dot product attention is used to measure the similarity between aspect word vectors + inference path and sentiment word vectors, as follows: (6) (7) In the formula, The similarity between representative word vectors + path vectors and sentiment word vectors. The dimension represented by the node. Representatives and words The set of nodes connected by information paths, i.e., the neighbor set of the C-edge, is then used by the commonsense knowledge graph to combine the syntax dependency information encoded by the D-edge with commonsense knowledge information. The combination yields a new node update function, which is: (8) In addition, a masking mechanism is used to obtain The aspect part indicates Then, average pooling is used to retain most of the information in the aspect representation, resulting in the final aspect representation vector of CPN. ,for: (9) In step 4, sentiment prediction is finally achieved through multi-task joint training, specifically as follows: This invention has two sub-tasks: aspect-level sentiment analysis and sentiment relationship recognition auxiliary task. The two models are trained together and share model parameters. The total loss of the model is the sum of the loss functions of the two sub-tasks. Aspect-level sentiment analysis tasks and their losses: For aspect-level sentiment analysis tasks, Feed to have In a fully connected network, the activation function is: (10) In the formula, and It consists of a trainable weight matrix and biases. These are the probabilities corresponding to the three emotions. For the loss within a given batch, the model uses cross-entropy loss and L2 regularization, as follows: (11) in, It is the tth real emotional tag. It is the t-th predicted sentiment label. It is the batch size for training. It is the regularization coefficient; Relationship identification auxiliary task and its loss: The emotional tone of the comment sentence itself is an important emotional element in the speaker's emotional expression. Capturing the emotional relationship between aspect words and the comment sentence can enhance the emotional connection. Therefore, for the relation recognition auxiliary task, the model concatenates the comment sentence vector and the aspect word vector as the relation recognition representation: (12) (13) In the formula, and It consists of a trainable weight matrix and biases. This refers to the probabilities of three relationships: aspect-level sentiment and sentence sentiment. For a given batch of loss, the model uses cross-entropy loss. (14) in, It is the first The true relationship labels between aspect-level sentiment and sentence sentiment in each sample. It is the first The predicted relationship between word sentiment and sentence sentiment labels for each sample aspect. This is the batch size for training; The model trains two classification tasks simultaneously and shares model parameters. The final training loss of the IASD-ML model mainly consists of three parts, and the overall loss function is defined as follows: (15) in, It is a level classification loss. It is a relation-level classification loss. It is a decision The hyperparameters for the relative importance weights.

[0033] The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0034] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for aspect-level sentiment analysis based on knowledge assistance and emotional relationship-driven approaches, characterized in that, include: Step 1: Define multiple tasks and construct the input representation; Step 2: Find and weight common sense paths based on external knowledge bases; Step 3: Construct a commonsense perception graph and update the node representations; Step 4: Through multi-task joint training, sentiment prediction is finally achieved.

2. The method according to claim 1, characterized in that, In step 1, multi-task definition is performed, specifically as follows: Define two subtasks, namely: Aspect-level sentiment classification task: Given a sentence and aspect word sequence For each aspect word, determine its emotional polarity, which includes positive, neutral, and negative. Sentiment Relationship Recognition Task: Define the relationship between aspect word sentiment and sentence sentiment tone, including: When the aspect term sentiment and the sentence sentiment are the same, the relationship is defined as similar; When emotions are opposite, the relationship is defined as opposite; A relationship is defined as irrelevant when only one of its emotions is neutral.

3. The method according to claim 2, characterized in that, In step 1, the input representation is constructed, specifically as follows: Pre-trained language model As an encoder, sentence-aspect word pairs are constructed and used as input to generate aspect-aware sentence representation vectors, which are then used as input representations.

4. The method according to claim 3, characterized in that, In step 2, common sense paths are searched and weighted based on external knowledge bases, specifically as follows: Use the SenticNet sentiment dictionary to identify sentiment words in sentences; Using aspect words as head nodes and identified sentiment words as tail nodes, common sense paths connecting them are searched in the common sense knowledge graph ConceptNet, and three information paths containing at least three entities are retained for each aspect word-sentiment word pair. Calculate the similarity between the sentence representation vector and the path representation, and use it as the path weight to obtain the weighted path representation.

5. The method according to claim 4, characterized in that, In step 3, a commonsense awareness graph is constructed and the node representations are updated, specifically as follows: Construct a commonsense perception graph based on sentence representation vectors and weighted path representations; The commonsense path network based on relational graph attention network updates the nodes in the commonsense perception graph.

6. The method according to claim 5, characterized in that, The commonsense path network based on the relational graph attention network updates the nodes in the commonsense perception graph, specifically as follows: Dependency node update: Aggregate the information of each word and its neighboring nodes on the syntactic dependency edges, and update the word representation to incorporate syntactic structure information; Common sense relationship node update: For nodes connected by common sense path edges, calculate the matching degree between aspect word vectors, path vectors and sentiment word vectors, and update the node representation accordingly; From the updated total node representations, extract the node representations corresponding to the aspect words, and generate the final aspect word representation vector through average pooling.

7. The method according to claim 6, characterized in that, In step 4, sentiment prediction is finally achieved through multi-task joint training, specifically as follows: The aspect word representation vector is input into a fully connected layer, the Softmax function is used to predict the sentiment polarity distribution of the aspect words, and the cross-entropy loss of the aspect-level sentiment classification task is calculated. The sentence representation vector and aspect word representation vector are concatenated and input into another fully connected layer to predict the relationship between aspect word sentiment and sentence sentiment tone, and to calculate the cross-entropy loss for the relationship recognition task. The losses of the two tasks are weighted and summed to obtain the total loss function of the model. The model is then trained end-to-end by minimizing the total loss. In the prediction phase, the trained model is used to output the sentiment polarity classification results of the aspect words.

Images