Attention mechanism and external knowledge-based aspect-level emotion discrimination method

Abstract

The invention belongs to the technical field of aspect category sentiment analysis, and particularly relates to an aspect level sentiment distinguishing method based on an attention mechanism and external knowledge. Aiming at the problems of inaccurate and incomplete aspect category sentiment analysis and the like, a text sequence is coded based on BERT and BiLSTM, and meanwhile, internal association between context and aspect categories is modeled by using specific BiLSTM, so that embedded representation of aspect category and context information fusion is formed; a common sense knowledge graph enhancement module is designed, a synonym set is searched for each token of an input text through WordNet / synonym forest, and expression obtained by specific BiLSTM is enriched through embedding of a knowledge graph; by designing a multi-aspect attention module, the correlation between each aspect category and vector representation containing context perception and knowledge perception is calculated, and deeper understanding and deduction of implicit emotion semantics of the text are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of aspect category sentiment analysis, and specifically relates to an aspect-level sentiment discrimination method based on attention mechanism and external knowledge. Background Art

[0002] Sentiment analysis is the process of analyzing people's emotions, attitudes, opinions, and sentiment expressions in textual reviews. Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment analysis task that includes many subtasks, in particular, aspect-class sentiment analysis (ACSA). The goal of aspect-class sentiment analysis is to predict the sentiment polarity towards a specific aspect category from the opinions expressed by the reviewer in a sentence. Unlike aspect-term sentiment analysis, aspect classes are usually more abstract and high-level than aspect terms, and they do not necessarily appear in a given text. Therefore, aspect-class sentiment analysis is a challenging task.

[0003] In aspect category sentiment analysis, insufficient semantic information in the text itself is an important challenge, which limits the model's modeling and deep understanding of sentiment semantics, and there are still defects in existing technologies. The difficulties of aspect category sentiment analysis are:

[0004] (1) Existing sentiment analysis studies usually rely on pipeline methods to solve the task. Obviously, the results of detecting aspect categories have a great impact on the performance of these methods. Errors in aspect class detection will affect sentiment polarity classification. In addition, these pipeline methods ignore the associations between two independent tasks, which are considered to be very important for the task.

[0005] (2) Aspect categories are usually abstract concepts that exist explicitly or implicitly in sentences, and the corresponding sentiment polarity is difficult to accurately identify. When irrelevant sentiment words have semantic meaning to aspect categories, aspect categories and sentiment words may not match. In addition, matching becomes difficult when sentiment is expressed implicitly.

[0006] Existing related methods or systems mainly focus on using pipeline methods to first identify the aspect categories contained in the sentence and then predict the polarity of the detected aspect categories. Although Lee et al. combined multi-task learning, the prediction of aspect categories and polarity in this method is still separate. With the development of deep learning and attention mechanism, several ACSA methods based on attention mechanism have emerged. Wang et al. proposed a long short-term memory network (LSTM) method based on attention mechanism, which can consider various parts of the text and take multiple aspect categories as input. Gao et al. further proposed a hierarchical attention network to solve the mismatch problem between aspects and sentiment words. Tay et al. incorporated aspect information into the model and learned to pay attention according to the relationship between aspects and sentence words. For multi-aspect sentiment analysis, Hu et al. proposed a constrained attention network for regulating attention and constraining attention weight allocation. In recent years, prompt fine-tuning has rapidly emerged and has been introduced into fine-grained sentiment analysis. For example, Gao et al. and Liu et al. studied fine-grained sentiment analysis tasks based on prompt fine-tuning and achieved promising results. Although these methods do not require tedious feature engineering, aspect categories and sentiment words may not match when irrelevant sentiment words have semantic meaning to aspect categories. In addition, matching becomes more difficult when sentiment is expressed implicitly.

[0007] Another popular class of methods is based on GNNs, which can assign more accurate sentiment information to aspect categories. Li et al. used graph attention networks to create representations for nodes in the parse tree of a given sentence. Cai et al. proposed a hierarchical graph convolutional network to identify multiple aspect categories in a given sentence and then predicted the sentiment polarity of each identified category. Zhang and Qian designed an interactive graph convolutional network for convolution on a hierarchical syntactic graph and a global vocabulary graph. Zhou et al. proposed a graph network model containing semantically relevant information of a graph convolutional network with edge gating. Liang et al. proposed a graph convolutional network that exploits the sentiment dependency of sentences according to specific aspects. However, most of these methods rely on the parse tree of sentences and sometimes cannot effectively parse sentences when the sentences contain implicit aspect categories.

[0008] In summary, existing aspect-level sentiment analysis techniques and models still have problems such as insufficient utilization of external knowledge and poor correlation between the two independent tasks of aspect class discrimination and sentiment discrimination. Therefore, a new method is needed to overcome these shortcomings in order to build a more stable, interpretable and superior aspect-level sentiment analysis method. Summary of the invention

[0009] In order to solve the problems of inaccurate and incomplete aspect category sentiment analysis caused by insufficient external knowledge utilization of text-based sentiment analysis methods, poor correlation between two independent tasks and difficulty in accurately detecting implicit aspect categories, the present invention enhances ACSA through carefully selected knowledge in the common sense knowledge graph, and provides an aspect-level sentiment discrimination method based on attention mechanism and external knowledge, which enables the model to obtain richer and more accurate emotions through multi-aspect attention mechanism and external knowledge.

[0010] To achieve the above object, the present invention adopts the following technical solutions:

[0011] The present invention provides an aspect-level sentiment discrimination method based on attention mechanism and external knowledge, comprising the following steps:

[0012] Get the set S of aspect-based sentiment texts, the text sequence X in the set S = (x1, x2, ..., x n ), n is the number of words;

[0013] The text sequence X is input into a pre-trained aspect-class sentiment analysis model, and the predicted aspect-class sentiment is output. The aspect-class sentiment analysis model is based on an attention mechanism and external knowledge, including a sequence encoding module, a common sense knowledge graph enhancement module, a gated fusion module, and a multi-aspect attention module.

[0014] Furthermore, the sequence encoding module integrates the aspect category information into the context information based on the sequence encoder of BERT and BiLSTM. In this way, the present invention comprehensively considers the internal association between text and aspect categories to alleviate the problem of mutual interference between multiple aspects of sentiment information, including:

[0015] Input layer: Use the pre-trained embedding BERT to embed the i-th word x in the text sequence X i Mapping to a continuous vector space E x ∈R v×d , and the output vector representation h i , where v is the size of the vocabulary, d is the embedding dimension, and the set of aspect categories is represented as a j is the jth aspect category, m is the number of aspect categories, and a random initialization method is used to represent the aspect vectors, which are fine-tuned during the training phase;

[0016] Shared BiLSTM layer: BiLSTM has been shown to be an effective way to incorporate contextual information into word embeddings. i Input into BiLSTM and represent the calculation of forward LSTM And the computational representation of backward LSTM Connect to get the context information fusion embedding representation h1i , as shown in formula (1):

[0017]

[0018] Where i = 1, 2, ..., n, is the computational representation of the forward LSTM. It is the computational representation of backward LSTM;

[0019] Specific BiLSTM layer: In the ACSA task, the aspect category information plays a guiding role. In the absence of a pre-given aspect category, each aspect category a in the aspect category set ASP is j Splice to h1 respectively i In the above example, BiLSTM is used to model the internal association between context and aspect categories, forming aspect category a j Embedded representation h2 fused with context information ij , as shown in formula (2):

[0020]

[0021] In the formula, i=1,2,…,n, j=1,2,…,m.

[0022] Furthermore, the common sense knowledge graph enhancement module is specifically:

[0023] In order to integrate external knowledge, based on existing work, the WordNet knowledge graph is introduced as an external knowledge base. He uses synonym sets to represent concepts and saves them in the form of (subject, relationship, object) triples. In the sentiment classification task, external knowledge plays a vital role, but the external common sense knowledge base usually contains a lot of noisy information. In order to find the most relevant knowledge embedding, bilinear operations and attention mechanisms are used for calculation. In this component, external knowledge is chosen to be encoded in a continuous vector space. Specifically, given any triple, it is hoped to learn the vector embeddings of the subject, relationship, and object in order to measure the effectiveness of the triple based on the embedding in the vector space.

[0024] Further integrating knowledge into BiLSTM is the core of this module. It takes BiLSTM representations as input and enriches them with relevant knowledge embeddings. The knowledge integration layer receives the relevant knowledge graph concept embeddings retrieved from the knowledge base and knowledge sentinels from the BiLSTM encoding layer.

[0025] Retrieve each tokenx in the text sequence X from the knowledge base i The candidate knowledge concept set C(x i ), for each concept c in the knowledge graph j ∈C(xi ) The corresponding embedding is expressed as Learning embedding representations in knowledge graphs using knowledge graph embedding models When the concept c in the knowledge graph j With TokenX i When relevant, the attention mechanism is used to select each tokenx i The most relevant knowledge concept is measured by bilinear operation to measure tokenx i and the concept c in the knowledge graph j Correlation, attention weight calculation is shown in formula (3):

[0026]

[0027] In the formula, is a trainable weight parameter;

[0028] When the concept c in the knowledge graph j With TokenX i When not relevant, introduce knowledge sentinels Calculate the attention weight as shown in formula (4):

[0029]

[0030] Using formula (3), the retrieved i All related knowledge concepts c j Perform weighted summation and use formula (4) to calculate the knowledge sentinel After weighting, the two representations are fused to obtain the vector hk with additional knowledge graph information i , calculated as shown in formula (5):

[0031]

[0032] When using the attention mechanism to select relevant knowledge concepts, the knowledge sentinel embedding will also be involved in the calculation. Specifically, the knowledge sentinel will calculate the attention weight together with other knowledge concepts. If the attention weight of the relevant knowledge concept of a token is low, the weight of the knowledge sentinel will be higher, playing a complementary role. j and Knowledge Sentinel Aligned to x i And aggregated into formula (5), through this external knowledge embedding, the global relevant knowledge of the text in the entire knowledge base can be easily integrated. And through this method, the representation is not only context-aware, but also knowledge-aware.

[0033] Furthermore, the gated fusion module combines the vector hk with the additional knowledge graph information through a fine-grained feature fusion method based on the gating mechanism.i Embedded representation h2 fused with context information ij Fusion, calculation is shown in equations (6) and (7):

[0034]

[0035] In the formula, represents the element-wise product operation, and g represents the gate calculated by the following formula:

[0036] g=σ(W g [h2 ij ;hk i ]+b g ) (7)

[0037] Where σ represents the sigmoid activation function, W g and b g are the parameters of this component, [h2 ij ;hk i ] is hk i and h2 ij connection.

[0038] Furthermore, the multi-aspect attention module is specifically:

[0039] For the ACSA task of a given aspect category, a single attention module is used to focus on the relevant information of the aspect category. For sentences containing multiple implicit aspect categories, the information capture ability of a single attention module may not be sufficient to meet the requirements of the task. Therefore, in each aspect category layer, a multi-aspect attention mechanism is used to enable the model to focus on different parts of the input during training, expand the representation space and enhance the information that needs to be paid attention to. For each aspect category layer, the attention layer input is the output of the gating mechanism h3 ij By calculating the aspect category a j and h3 ij The correlation of ij The weight is the state h3 ij In terms of inference, category a j The linear shift of the vector can capture the relationship between the two words, so subtractive attention is used to capture the aspect category a j and h3 ij The correlation is shown in formula (8):

[0040] score(h3 ij ; a j )=V T tanW((h3 ij -a j )+b) (8)

[0041] Where V T represents a learnable weight matrix, tan represents the hyperbolic tangent function, which is an activation function that compresses the result of the linear transformation in W to between (-1,1), and b is the bias term;

[0042] Each aspect category uses the score function of formula (8) to calculate the attention weight a ij and fusion representation As shown in equations (9) and (10):

[0043]

[0044] Use formula (9) multiple times to build multiple attention modules, and get l is the number of times formula (9) is used, and the lth attention module is used to obtain the lth fusion representation after fusion;

[0045] Will Connect and project to construct the corresponding aspect category a j The final expression of is shown in formula (11):

[0046]

[0047] In the formula, Indicates that j The corresponding representation of the lth attention module, W j1 is the parameter matrix;

[0048] Mult j After linear transformation, it is input into the fully connected layer classifier to obtain the predicted aspect category emotion probability distribution vector, as shown in formula (12):

[0049] y j =softmax(Mult j ·W j2 +b j ) (12)

[0050] Where W j2 and b j are the weights and biases of the fully connected layer;

[0051] The output results of multiple emotion labels are obtained, as shown in formula (13):

[0052] y=[y1;y2;...;y m ] (13)

[0053] Where m represents the number of aspect categories.

[0054] For the entire sentence, we need to predict the sentiment of each aspect category accordingly. We design a joint training strategy to combine multiple classifiers and transform the ACSA task into a multi-label classification task. j , use formula (12) to obtain the representation of the aspect level attention layer, and then input the result into the fully connected layer for use in the sentiment classifier. The sentiment polarity classification result of the aspect category is determined by the label with the highest probability obtained by formula (12). By using formula (12), multiple classifiers can be jointly trained, and the output results of multiple sentiment labels can be obtained by formula (13).

[0055] Furthermore, the method further includes orthogonal regularization, applying orthogonal regularization to the multi-aspect attention module, and calculating orthogonal regularization R0, as shown in formula (14):

[0056] R0=||M T MI|| (14)

[0057] Where M∈R represents a two-dimensional attention matrix.

[0058] Furthermore, the method also includes training of aspect-based sentiment analysis models, specifically:

[0059] Define the weight calculation of sentiment tags, as shown in (15) and (16):

[0060]

[0061] In the formula, Is aspect category a j The number of sentiment labels k in b jk is a j The weight of the sentiment label k in Yes jk The normalized representation of ;

[0062] The objective function of training is to minimize the cross entropy loss function, as shown in formula (17):

[0063]

[0064] In the formula, y jk is the aspect category a in a given sentence j The true value of the sentiment label, is the predicted aspect category a j The sentiment label of , L is the final objective function.

[0065] Furthermore, in order to exclude the uninvolved aspect categories, the sentiment labels are set to four types {-2, -1, 0, 1}, where -2, -1, 0, 1 represent no, negative, neutral, and positive sentiments, respectively.

[0066] Compared with the prior art, the present invention has the following advantages:

[0067] (1) The text sequence is encoded based on BERT and BiLSTM, and a specific BiLSTM is used to model the internal association between context and aspect categories to form an embedded representation that fuses aspect categories and context information, thereby solving the problem of insufficient context information representation capability for aspect-level sentiment tasks.

[0068] (2) Design a common sense knowledge graph enhancement module to search for synonyms for each token in the input text through WordNet / Synonym Dictionary, and use the embedding of the knowledge graph to enrich the representation obtained by a specific BiLSTM, providing more comprehensive knowledge semantics and alleviating the lack of limited annotated data.

[0069] (3) By designing a multi-aspect attention layer and calculating the correlation between each aspect category and the context-aware and knowledge-aware vector representation, we can achieve a deeper understanding and inference of the implicit sentiment semantics of the text, thereby improving the effect of aspect category sentiment analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a framework diagram of the sentiment analysis model of the present invention. DETAILED DESCRIPTION

[0071] In order to further illustrate the technical solution of the present invention, the present invention is further described below through embodiments.

[0072] The present embodiment provides an aspect-level sentiment discrimination method based on an attention mechanism and external knowledge, comprising the following steps:

[0073] Get the set S of aspect-based sentiment texts, the text sequence X in the set S = (x1, x2, ..., x n ), n is the number of words, such as: X = "Students and local residents squeezed into this narrow space to eat cheap and delicious fast food.", sentence X corresponds to the display aspect category c x = "price", corresponding to sentiment y i = "positive"; implicit aspect category c x = "food, atmosphere", corresponding to emotion y i = "positive, negative";

[0074] The text sequence X is input into the pre-trained aspect-based sentiment analysis model, and the predicted aspect-based sentiment is output. The aspect-based sentiment analysis model is based on the attention mechanism and external knowledge, and its framework is as follows: Figure 1As shown, it includes a sequence encoding module, a common sense knowledge graph enhancement module, a gated fusion module and a multi-aspect attention module.

[0075] 1) Sequence encoding module:

[0076] The BERT- and BiLSTM-based sequence encoder integrates aspect category information into context information, including the input layer, shared BiLSTM layer, and specific BiLSTM layer.

[0077] Input layer: For the text sequence X = "Students and local residents squeeze into this narrow space to eat cheap and delicious fast food.", use the pre-trained embedding BERT to embed the i-th word x in the text sequence X i Mapping to a continuous vector space E x ∈R v×d , and the output vector representation h i , where v is the size of the vocabulary, d is the embedding dimension, and the set of aspect categories is represented as a j is the jth aspect category, m is the number of aspect categories;

[0078] Shared BiLSTM layer: The vector representation h i Input into BiLSTM and represent the calculation of forward LSTM And the computational representation of backward LSTM Connect to get the context information fusion embedding representation h1 i , as shown in formula (1):

[0079]

[0080] Where i = 1, 2, ..., n, is the computational representation of the forward LSTM. It is the computational representation of backward LSTM;

[0081] Specific BiLSTM layer: each aspect category a in the aspect category set ASP j Splice to h1 respectively i In the formation of aspect category a j Embedded representation h2 fused with context information ij , as shown in formula (2):

[0082]

[0083] In the formula, i=1,2,…,n, j=1,2,…,m.

[0084] 2) Common sense knowledge graph enhancement module:

[0085] Retrieve each tokenx in the text sequence X from the knowledge base i The candidate knowledge concept set C(x i ), for each concept c in the knowledge graph j ∈C(x i ) The corresponding embedding is expressed as Learning embedding representations in knowledge graphs using knowledge graph embedding models When the concept c in the knowledge graph j With TokenX i When relevant, the attention mechanism is used to select each tokenx i The most relevant knowledge concept is measured by bilinear operation to measure tokenx i and the concept c in the knowledge graph j Correlation, attention weight calculation is shown in formula (3):

[0086]

[0087] In the formula, is a trainable weight parameter;

[0088] When the concept c in the knowledge graph j With TokenX i When not relevant, introduce knowledge sentinels Calculate the attention weight as shown in formula (4):

[0089]

[0090] Using formula (3), the retrieved i All related knowledge concepts c j Perform weighted summation and use formula (4) to calculate the knowledge sentinel After weighting, the two representations are fused to obtain the vector hk with additional knowledge graph information i , calculated as shown in formula (5):

[0091]

[0092] 3) Gated fusion module:

[0093] The vector hk with additional knowledge graph information is fused by a fine-grained feature fusion method based on a gating mechanism. i Embedded representation h2 fused with context information ij Fusion, calculation is shown in equations (6) and (7):

[0094]

[0095] In the formula, represents the element-wise product operation, and g represents the gate calculated by the following formula:

[0096] g=σ(W g [h2 ij ;hk i ]+b g ) (7)

[0097] Where σ represents the sigmoid activation function, W g and b g are the parameters of this component, [h2 ij ;hk i ] is hk i and h2 ij connection.

[0098] 4) Multi-aspect Attention Module:

[0099] Use subtractive attention to capture aspect category a j and h3 ij The correlation is shown in formula (8):

[0100] score(h3 ij ; a j )=V T tanW((h3 ij -a j )+b) (8)

[0101] Where V T represents a learnable weight matrix, tan represents the hyperbolic tangent function, which is an activation function that compresses the result of the linear transformation in W to between (-1, 1), and b is the bias term;

[0102] Each aspect category uses the score function of formula (8) to calculate the attention weight a ij and fusion representation As shown in equations (9) and (10):

[0103]

[0104] Use formula (9) multiple times to build multiple attention modules, and get l is the number of times formula (9) is used, and the lth attention module is used to obtain the lth fusion representation after fusion;

[0105] Will Connect and project to construct the corresponding aspect category a j The final expression of is shown in formula (11):

[0106]

[0107] In the formula, Indicates that j The corresponding representation of the lth attention module, W j1 is the parameter matrix;

[0108] Mult j After linear transformation, it is input into the fully connected layer classifier to obtain the predicted aspect category emotion probability distribution vector, as shown in formula (12):

[0109] y j =softmax(Mult j ·W j2 +b j ) (12)

[0110] Where W j2 and b j are the weights and biases of the fully connected layer;

[0111] The output results of multiple emotion labels are obtained, as shown in formula (13):

[0112] y=[y1;y2;...;y m ] (13)

[0113] Where m represents the number of aspect categories.

[0114] As a further optimization, this method also includes orthogonal regularization, which forces the attention weight vectors of different aspect categories to be orthogonal so that different aspect categories focus on different parts of a given sentence and minimize overlap. Apply orthogonal regularization to the multi-aspect attention module. Let X be a sentence. Assume that X contains non-overlapping aspect categories and calculate the orthogonal regularization R0, as shown in formula (14):

[0115] R0=||M T MI|| (14)

[0116] Where M∈R represents a two-dimensional attention matrix.

[0117] As a further optimization, the method also includes the training of the aspect-based sentiment analysis model (model optimization), taking into account the imbalance problem in the data and considering the different costs caused by the misclassification of samples of different aspect categories to balance the losses. In order to obtain a high reward (penalty) when predicting the correct (wrong) polarity of the sentiment, the weight calculation is defined using formulas (15) to (16). Therefore, during the training process, the model focuses on optimizing samples with the correct sentiment label.

[0118] Define the weight calculation of sentiment tags, as shown in (15) and (16):

[0119]

[0120]

[0121] In the formula, Is aspect category a j The number of sentiment labels k in b jk is a j The weight of the sentiment label k in Yes jk The normalized representation of ;

[0122] The objective function of training is to minimize the cross entropy loss function, as shown in formula (17):

[0123]

[0124] In the formula, y jk is the aspect category a in a given sentence j The true value of the sentiment label, is the predicted aspect category a j The sentiment label of , L is the final objective function.

[0125] In order to exclude uninvolved aspect categories, the sentiment labels are set to four types {-2, -1, 0, 1}, where -2, -1, 0, 1 represent no, negative, neutral, and positive sentiments, respectively.

[0126] It should be noted that the implementation of this model is based on the PyTorch framework, the operating environment is Ubuntu 22.04, and the computing device is NVIDIA TITAN RTX. The detailed experimental configuration and hyperparameter settings during the model learning process are shown in Table 1 below.

[0127] Table 1 Model parameter configuration table

[0128]

[0129] The present invention is compared with a series of current international cutting-edge sentiment analysis and representation classification methods, such as AT-LSTM-CAV / CAM, MPADB, SRGN-BERT, BERT-pair-QA-B SCAPT, AC-MIMLLN-BERT. Experiments were conducted on three aspect category sentiment analysis data sets, and the present invention uses the accuracy ACC to identify the performance of aspect category sentiment classification. The comparison results are shown in Table 2 below.

[0130] Table 2 Comparison of experimental results

[0131]

[0132] It can be seen from Table 2 that the accuracy of the proposed model in the three categories of sentiment analysis data sets is better than that of other models.

[0133] Through detailed comparison and ablation experiments on sentiment analysis datasets of three categories, the method proposed in this paper has the following significant advantages:

[0134] (1) Full utilization of external knowledge information: Compared with the most advanced models, the method proposed in this invention has the best effect on both Dseme14 and the more difficult DMAMS-ACSA datasets. This shows that the model can effectively identify aspect categories and emotions in text sequences. In addition, the ablation experiment analysis based on the semantic enhancement of external knowledge verifies the effective utilization of external knowledge by the present invention. The semantic enhancement of the knowledge graph provides significant semantic expansion and information supplementation effects by integrating the synonym set of the input sentence, and we first learn its embedding considering the entire knowledge graph, and then integrate the learned embedding retrieval into ACSA. In this way, these representations not only consider the local context on individual knowledge points, but also utilize all known facts and relationships in the knowledge base to make the knowledge more informative.

[0135] (2) More accurate detection of implicit aspect categories and direct modeling of the relationship between aspect categories and corresponding opinion words: First, by integrating knowledge information and contextual semantic information, an enhanced text representation is generated, enabling the model to better understand and capture the relationship between aspect categories and sentiment opinion words. Second, a multi-aspect attention mechanism is used to form an aspect category a j The embedded representation fused with context information expands the representation space and enhances the information that needs attention. Through experiments on aspect category sentiment analysis datasets, the present invention can effectively improve the performance of the model in sentiment analysis tasks.

[0136] Therefore, the method proposed in the present invention has significant advantages in fully utilizing external knowledge information and detecting implicit aspect categories and directly modeling the relationship between aspect categories and corresponding opinion words.

[0137] The above shows and describes the main features and advantages of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present invention.

[0138] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for aspect-level sentiment discrimination based on attention mechanism and external knowledge, characterized in that: The following steps are involved: Get the set S of aspect-type sentiment texts, the text sequence X in the set S = (x1, x2, ..., x n ), n is the number of words; The text sequence X is input into a pre-trained aspect-class sentiment analysis model, and the predicted aspect-class sentiment is output. The aspect-class sentiment analysis model is based on an attention mechanism and external knowledge, including a sequence encoding module, a common sense knowledge graph enhancement module, a gated fusion module, and a multi-aspect attention module.

2. According to claim 1, the aspect-level sentiment discrimination method based on attention mechanism and external knowledge is characterized in that: The sequence encoding module is based on the sequence encoder of BERT and BiLSTM, which integrates the aspect category information into the context information, including: Input layer: Use the pre-trained embedding BERT to embed the i-th word x in the text sequence X i Mapping to a continuous vector space E x ∈R v×d , and the output vector representation h i , where v is the size of the vocabulary and d is the embedding dimension. The set of aspect categories is represented as Among them, a j is the jth aspect category, m is the number of aspect categories; Shared BiLSTM layer: The vector representation h i Input into BiLSTM and represent the calculation of forward LSTM And the computational representation of backward LSTM Connect to get the context information fusion embedding representation h1 i , as shown in formula (1): Where i = 1, 2, ..., n, is the computational representation of the forward LSTM. It is the computational representation of backward LSTM; Specific BiLSTM layer: each aspect category a in the aspect category set ASP j Splice to h1 respectively i In the formation of aspect category a j Embedded representation h2 fused with context information ij , as shown in formula (2): In the formula, i=1,2,…,n, j=1,2,…,m.

3. According to claim 2, the aspect-level sentiment discrimination method based on attention mechanism and external knowledge is characterized in that: The common sense knowledge graph enhancement module is specifically: Retrieve each tokenx in the text sequence X from the knowledge base i The candidate knowledge concept set C(x i ), for each concept c in the knowledge graph j ∈C(x i ) The corresponding embedding is expressed as Learning embedding representations in knowledge graphs using knowledge graph embedding models When the concept c in the knowledge graph j With TokenX i When relevant, the attention mechanism is used to select each tokenx i The most relevant knowledge concept is measured by bilinear operation to measure tokenx i and the concept c in the knowledge graph j Correlation, attention weight calculation is shown in formula (3): In the formula, is a trainable weight parameter; When the concept c in the knowledge graph j With TokenX i When not relevant, introduce knowledge sentinels Calculate the attention weight as shown in formula (4): Using formula (3), the retrieved i All related knowledge concepts c j Perform weighted summation and use formula (4) to calculate the knowledge sentinel After weighting, the two representations are fused to obtain the vector hk with additional knowledge graph information i , calculated as shown in formula (5):

4. According to claim 3, the aspect-level sentiment discrimination method based on attention mechanism and external knowledge is characterized in that: The gated fusion module combines the vector hk with additional knowledge graph information through a fine-grained feature fusion method based on the gating mechanism. i Embedded representation h2 fused with context information ij Fusion, calculation is shown in equations (6) and (7): In the formula, represents the element-wise product operation, and g represents the gate calculated by the following formula: g=σ(W g [h2 ij ;hk i ]+b g ) (7) Where σ represents the sigmoid activation function, W g and b g are the parameters of this component, [h2 ij ;hk i ] is hk i and h2 ij connection.

5. According to claim 4, the aspect-level sentiment discrimination method based on attention mechanism and external knowledge is characterized in that: The multi-aspect attention module is specifically: Use subtractive attention to capture aspect categories a j and h3 ij The correlation is shown in formula (8): score(h3 ij ;has j )=V T tanW((h3 ij -has j )+b) (8) Where V T represents a learnable weight matrix, tan represents the hyperbolic tangent function, which is an activation function that compresses the result of the linear transformation in W to between (-1, 1), and b is the bias term; Each aspect category uses the score function of formula (8) to calculate the attention weight a ij and fusion representation As shown in equations (9) and (10): Use formula (9) multiple times to build multiple attention modules, and get l is the number of times formula (9) is used, and the lth attention module is used to obtain the lth fusion representation after fusion; Will Connect and project to construct the corresponding aspect category a j The final expression of is shown in formula (11): In the formula, Indicates that j The corresponding representation of the lth attention module, W j1 is the parameter matrix; Mult j After linear transformation, it is input into the fully connected layer classifier to obtain the predicted aspect category emotion probability distribution vector, as shown in formula (12): y j =softmax(Mult j ·W j2 +b j ) (12) Where W j2 and b j are the weights and biases of the fully connected layer; The output results of multiple emotion labels are obtained, as shown in formula (13): y=[y1;y2;...;y m ] (13) Where m represents the number of aspect categories.

6. The aspect-level sentiment discrimination method based on attention mechanism and external knowledge according to claim 1 is characterized in that: The method further includes orthogonal regularization, applying orthogonal regularization to the multi-aspect attention module, and calculating the orthogonal regularization R0, as shown in formula (14): R0=||M T M-I|| (14) Where M∈R represents a two-dimensional attention matrix.

7. The aspect-level sentiment discrimination method based on attention mechanism and external knowledge according to claim 1 is characterized in that: The method also includes training an aspect-based sentiment analysis model, specifically: Define the weight calculation of sentiment tags, as shown in (15) and (16): In the formula, Is aspect category a j The number of sentiment labels k in b jk is a j The weight of the sentiment label k in Yes jk The normalized representation of ; The objective function of training is to minimize the cross entropy loss function, as shown in formula (17): In the formula, y jk is the aspect category a in a given sentence j The true value of the sentiment label, is the predicted aspect category a j The sentiment label of , L is the final objective function.

8. The aspect-level sentiment identification method based on attention mechanism and external knowledge according to claim 6 or 7, characterized in that: The sentiment labels are set to four types {-2, -1, 0, 1}, where -2, -1, 0, 1 represent no, negative, neutral, and positive sentiments, respectively.

Images