A dual-objective entity sentiment analysis method based on multi-task learning
Through the dual-object entity emotion analysis method of multi-task learning, the semantic representation of emotional sentences is solved by using context breakers to identify and separate emotional sentences, and the problem of unrelated emotional expression in aspect-level emotion analysis is solved, which significantly improves the performance and accuracy of emotion analysis. .
Patent Information
- Application Number
- CN202210054948.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-18
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-01-18
AI Technical Summary
In aspect-level sentiment analysis, it is difficult for the prior art to effectively distinguish and suppress irrelevant affective expressions between different target entities, resulting in increased complexity of sentiment analysis.
Using the dual-object entity sentiment analysis method of multi-task learning, the dual-objective entity sentiment analysis is carried out through joint training of sentence context breaking symbol recognition and classification of left and right entities emotion polarity. The BERT language model is used to identify context breaking symbols and separate the semantic representation of emotional sentences, and perform dual-objectives. Analysis of emotional polarity of entities.
It effectively overcomes the influence of irrelevant emotional expression in aspect-level emotion analysis, improves the performance of dual-target entity sentiment analysis, and can be applied to various types of aspect-level emotion analysis tasks.
Smart Images

Figure CN115017912B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to aspect-level sentiment analysis in natural language understanding, and specifically to a multi-task learning dual-target entity sentiment analysis method, which can be widely used in aspect-level sentiment analysis tasks in various fields. Background Art
[0002] The purpose of aspect-level sentiment classification is to predict the polarity of multiple target entities in a sentence or document. It is a fine-grained sentiment analysis task. Different from traditional sentiment analysis tasks, it performs polarity analysis on target entities (generally classified into three categories: positive, negative, and neutral). Aspect-level sentiment classification is often used in reviewers' review sentences, such as shopping reviews, restaurant reviews, and movie reviews. Aspect-level sentiment classification usually has two aspect words and their related sentiment orientations in one sentence. For example, the sentence "Prices are higher to dine but their food is quite good" is negative for the target entity "Prices", but positive for the target entity "food".
[0003] With the continuous development of artificial neural network technology, various neural networks such as Long Short-Term Memory (LSTM), Deep Memory Network and Bidirectional Encoder Representations from Transformers (BERT) language model proposed by Google AI Language have been applied to aspect polarity classification, thus providing an end-to-end classification method for it without any feature engineering work. However, when there are multiple target entities in a sentence, the aspect polarity classification task needs to distinguish different aspects of emotions. Therefore, compared with document-level sentiment analysis with only one overall sentiment orientation, the aspect polarity classification task is more complex. The main challenge is: when performing sentiment analysis on different target entities, how to highlight the emotional expressions related to it and suppress the emotional expressions that are not related to it. In order to achieve this goal, a variety of aspect-centric sentiment semantic learning methods have been proposed for deep learning methods for aspect polarity classification, such as attention-based semantic learning, position decay, left-right semantic learning, aspect connection and global semantic learning, etc., but each method has a certain degree of influence of irrelevant emotional expressions. In order to completely solve the influence of irrelevant emotional expressions in multi-target sentiment analysis, the present invention proposes a dual-target entity sentiment analysis method based on multi-task learning, which separates the emotional expressions of two target entities in a sentiment sentence from each other through contextual delimiters. Summary of the invention
[0004] The present invention discloses a dual-target entity sentiment analysis method of multi-task learning. Through multi-task learning of sentence context delimiter recognition and left and right entity sentiment polarity classification, a neural network model with automatic sentence context delimiter recognition and dual-target entity sentiment polarity automatic classification is jointly trained to solve the aspect-level sentiment analysis problem in a more effective way.
[0005] To achieve the above object, the technical solution of the present invention is:
[0006] A dual-target entity sentiment analysis method based on multi-task learning, characterized by comprising the following steps:
[0007] S1. Through multi-task learning of sentence context delimiter recognition and left and right entity sentiment polarity classification, a neural network model with automatic sentence context delimiter recognition and dual-target entity sentiment polarity automatic classification is jointly trained;
[0008] S2. Using the neural network model trained in step S1 to identify contextual delimiters in the sentiment sentence;
[0009] S3. In the neural network model trained in step S1, the semantic representation of the sentiment sentence is separated according to the position corresponding to the context separator obtained in step S2 to obtain the left clause semantic representation and the right clause semantic representation, and then sentiment analysis is performed on the left clause semantic representation and the right clause semantic representation respectively, and finally the sentiment polarity of the dual target entities is obtained;
[0010] The emotional sentence refers to a multi-emotional expression sentence containing two target entities, left and right;
[0011] The context separator refers to a word located between the left and right target entities in the emotional sentence, which separates the emotional expressions of the two target entities from each other;
[0012] The neural network model refers to a neural network structure based on the BERT language model; the BERT language model refers to the Bidirectional Encoder Representations from Transformers (BERT) language model proposed by Google AI Language.
[0013] Furthermore, the step S1 specifically includes:
[0014] S1.1 The input sequence s of the BERT language model is composed of sentiment sentences Sen = {…, t1, w1, w2,…, w n ,t2,…} and BERT encoding symbols, as shown below:
[0015]
[0016] Mid={w1,n2,...,w n} (2)
[0017] Where [CLS] is the encoding of the BERT classifier, [SEP] is the encoding of the BERT terminator, t1 is the left target entity to be analyzed, t2 is the right target entity to be analyzed, Mid = {w1, w2, ..., w n} is the middle word sequence between the left and right target entities t1 and t2, “…” represents the omitted word sequence, m is the length of the input sequence s, d w is the dimension of character encoding in BERT, n is the length of the middle word sequence Mid, and the "word" refers to the language segment separated by Bert's tokenizer Tokenzier;
[0018] S1.2 Send the input sequence s to the BERT language model for processing to obtain the sentence semantic representation C of the emotional sentence Sen Sen , as shown below:
[0019]
[0020] Among them, BERT(·) represents the BERT language model, is the i-th hidden state of the BERT language model, d b is the number of hidden units of the BERT language model;
[0021] S1.3 According to the corresponding relationship, from C Sen Extract the middle word sequence Mid={w1,w2,...,w n The intermediate semantics corresponding to C Mid , as shown below:
[0022]
[0023] Among them, Middle(·) represents intermediate semantic extraction, is the i-th middle word w i In C Sen The corresponding hidden state in ;
[0024] S1.4 Intermediate semantic representation C Mid Perform a softmax linear transformation to identify contextual delimiters. The calculation process is as follows:
[0025] γ=C Mid ·G+b1 (5)
[0026]
[0027] in, is a learnable parameter vector for contextual delimiter recognition, b1∈R is a bias parameter, “·” represents the dot product operation of the vector, and γ∈R n is the context segmentation confidence score vector corresponding to the middle word sequence Mid, w is an middle word, P(w|C Mid ,θ) represents the predicted probability that the middle word w is a context separator, It means to return the value that makes P(w|C Mid ,θ) is the maximum value of the middle word, w * is the calculated contextual delimiter, θ is the set of all learnable parameters, and exp(·) represents the exponential function with e as the base;
[0028] S1.5 in w sp As a separator, two mask matrices consisting of 1 and 0 are formed to represent the sentence semantics C Sen Separated into left clause semantic representation C left and the semantic representation of the right clause C right , the calculation process is as follows:
[0029]
[0030] Among them, mask L is the mask matrix used to separate the semantics of the left clause, mask r is the mask matrix used to separate the semantics of the right clause, is a vector of all 1s, is a vector of all zeros, tonken i ∈Sen is the i-th word in the sentence Sen, and the function Idx(·) finds the position number of the specified word in the sentence Sen. For mask L The i-th column vector in , i∈[1,m] and is an integer, For mask r The j-th column vector in , j∈[1,m] is an integer, and “⊙” represents element-by-element multiplication;
[0031] S1.6 respectively in the left clause semantic representation C left and the right clause semantics C right A multi-head self-attention encoding process is performed on it to obtain the semantic encoding C′ of the left clause left and the semantic encoding of the right clause C′ right , the calculation process is as follows:
[0032]
[0033] Among them, MHSA(X) represents the input Multi-head attention MHA(Q,K,V);
[0034] S1.7 Semantically encode the left clause C′ left and the semantic encoding of the right clause C′ right Perform average pooling operation to obtain the left clause sentiment vector Z L and the right clause sentiment vector Z r , the calculation process is as follows:
[0035]
[0036] Among them, avePooling(C) represents the parameter Perform a column-wise average pooling operation;
[0037] S1.8 respectively for the left clause sentiment vector Z L and the right clause sentiment vector Z r Perform a linear transformation of softmax, calculate the probability of sentiment polarity, and obtain the final sentiment polarity. The calculation process is as follows:
[0038] u L =Q·Z L +b2 (18)
[0039]
[0040] u r =Q·Z r +b2 (21)
[0041]
[0042] in, is the representation matrix of sentiment polarity, is a bias vector, d k is the number of sentiment polarity categories, Y is the set of sentiment polarity categories, y is a sentiment polarity, Z L and Z r The corresponding sentiment polarity confidence score vector, P(y|Z L ,θ)、P(y|Z r ,θ) represent Z L and Z r The predicted probability of sentiment polarity y, y L ,y r They are the left and right emotional polarities of the final assessment, They represent the return value such that P(y|Z L ,θ) and P(y|Z r,θ) is the maximum sentiment polarity, θ is the set of all learnable parameters, and exp(·) represents the exponential function with e as the base.
[0043] Furthermore, in step S1, a joint training method for jointly training a neural network model with automatic recognition of sentence context delimiters and automatic classification of dual-target entity sentiment polarities is:
[0044] (1) The cross entropy loss error is used to calculate the loss function of context segmentation symbol recognition and the loss function of dual-target entity sentiment analysis. The calculation process is as follows:
[0045]
[0046] Among them, Ω is the set of training sentences for the dual-target entity sentiment analysis task, |Ω| represents the size of the set Ω, is the word label of the context separator of the i-th training sentence in Ω, is the intermediate semantic representation of the i-th training sentence in Ω, are the left sentiment polarity label and the right sentiment polarity label of the i-th training sentence in Ω, are the left clause sentiment vector and the right clause sentiment vector of the i-th training sentence in Ω, respectively, Mid (θ) is the loss function used in contextual delimiter recognition training, Ψ L (θ) is the loss function used in the left target entity sentiment analysis training, Ψ r (θ) is the loss function used when training the right target entity sentiment analysis;
[0047] (2) Use the following formula (27) to calculate the joint loss function for joint training of sentence context segmentation symbol recognition and dual-target entity sentiment polarity classification:
[0048]
[0049] Among them, α1 and α2 are two weight parameters;
[0050] (3) The joint training goal is to minimize the joint loss error calculated by formula (27).
[0051] In order to completely solve the influence of irrelevant emotional expressions in multi-target sentiment analysis, the present invention proposes a dual-target entity sentiment analysis method of multi-task learning, which separates the emotional expressions of two target entities in the emotional sentence through contextual delimiters. First, through the joint learning of sentence context delimiter recognition and left and right entity emotional polarity classification, a neural network model with automatic sentence context delimiter recognition and dual-target entity emotional polarity automatic classification is jointly trained. Secondly, the trained neural network model is used to identify the context delimiters in the emotional sentence. Furthermore, the semantic representation of the emotional sentence is separated according to the position corresponding to the obtained context delimiter to obtain the left clause semantic representation and the right clause semantic representation, and then the left clause semantic representation and the right clause semantic representation are respectively subjected to sentiment analysis to obtain the sentiment polarity of the dual-target entities.
[0052] The present invention has the following advantages:
[0053] (1) Through a BERT language model that is extensively pre-trained and fine-tuned for tasks, emotional sentences are dynamically encoded, which can effectively overcome the problem of too small a corpus for aspect-level sentiment analysis;
[0054] (2) The context separator is used to separate the emotional expressions of the two target entities in the emotional sentence, which completely solves the influence of irrelevant emotional expressions in multi-target sentiment analysis;
[0055] (3) The dual-target entity sentiment analysis is converted into two independent single-target entity sentiment analyses through contextual delimiters, which greatly improves the performance of dual-target entity sentiment analysis;
[0056] (4) By converting a sentiment sentence containing more target entities into multiple dual-target entity sentiment sentences, the method of the present invention can be applied to various types of aspect-level sentiment analysis tasks. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 It is a schematic flow chart of the method of the present invention. DETAILED DESCRIPTION
[0058] The present invention is further described below in conjunction with specific embodiments, but the protection scope of the present invention is not limited to the following embodiments.
[0059] Suppose the sentiment sentence Sen contains the left target entity t1 and the right target entity t2 = {…, t1, w1, w2…, w n ,t2,…}, the sentiment of the two target entities t1 and t2 is analyzed by the following steps:
[0060] S1. Through multi-task learning of sentence context delimiter recognition and left and right entity sentiment polarity classification, a neural network model with automatic sentence context delimiter recognition and dual-target entity sentiment polarity automatic classification is jointly trained;
[0061] S2. Using the neural network model trained in step S1 to identify contextual delimiters in the sentiment sentence;
[0062] S3. In the neural network model trained in step S1, the semantic representation of the sentiment sentence is separated according to the position corresponding to the context separator obtained in step S2 to obtain the left clause semantic representation and the right clause semantic representation, and then sentiment analysis is performed on the left clause semantic representation and the right clause semantic representation respectively, and finally the sentiment polarity of the dual target entities is obtained;
[0063] The emotional sentence refers to a multi-emotional expression sentence containing two target entities, left and right;
[0064] The context separator refers to a word located between the left and right target entities in the emotional sentence, which separates the emotional expressions of the two target entities from each other;
[0065] The neural network model refers to a neural network structure based on the BERT language model; the BERT language model refers to the Bidirectional Encoder Representations from Transformers (BERT) language model proposed by Google AI Language.
[0066] Furthermore, the step S1 specifically includes:
[0067] S1.1 The input sequence s of the BERT language model is composed of sentiment sentences Sen = {…, t1, w1, w2,…, w n ,t2,…} and BERT encoding symbols, as shown below:
[0068]
[0069] Mid={w1,w2,...,w n} (2)
[0070] Where [CLS] is the encoding of the BERT classifier, [SEP] is the encoding of the BERT terminator, t1 is the left target entity to be analyzed, t2 is the right target entity to be analyzed, Mid = {w1, w2, ..., w n} is the middle word sequence between the left and right target entities t1 and t2, “…” represents the omitted word sequence, m is the length of the input sequence s, d wis the dimension of character encoding in BERT, n is the length of the middle word sequence Mid, and the "word" refers to the language segment separated by Bert's tokenizer Tokenzier;
[0071] S1.2 Send the input sequence s to the BERT language model for processing to obtain the sentence semantic representation C of the emotional sentence Sen Sen , as shown below:
[0072]
[0073] Among them, BERT(·) represents the BERT language model, is the i-th hidden state of the BERT language model, d b is the number of hidden units of the BERT language model;
[0074] S1.3 According to the corresponding relationship, from C Sen Extract the middle word sequence Mid={w1,w2,...,w n The intermediate semantics corresponding to C Mid , as shown below:
[0075]
[0076] Among them, Middle(·) represents intermediate semantic extraction, is the i-th middle word w i In C Sen The corresponding hidden state in ;
[0077] S1.4 Intermediate semantic representation C Mid Perform a softmax linear transformation to identify contextual delimiters. The calculation process is as follows:
[0078] γ=C Mid ·G+b1 (5)
[0079]
[0080] in, is a learnable parameter vector for contextual delimiter recognition, b1∈R is a bias parameter, “·” represents the dot product operation of the vector, and γ∈R n is the context segmentation confidence score vector corresponding to the middle word sequence Mid, w is an middle word, P(w|C Mid ,θ) represents the predicted probability that the middle word w is a context separator, It means to return the value that makes P(w|C Mid ,θ) is the maximum value of the middle word, w *is the calculated contextual delimiter, θ is the set of all learnable parameters, and exp(·) represents the exponential function with e as the base;
[0081] S1.5 in w sp As a separator, two mask matrices consisting of 1 and 0 are formed to represent the sentence semantics C Sen Separated into left clause semantic representation C left and the right clause semantics C right , the calculation process is as follows:
[0082]
[0083] Among them, mask L is the mask matrix used to separate the semantics of the left clause, mask r is the mask matrix used to separate the semantics of the right clause, is a vector of all 1s, is a vector of all zeros, tonken i ∈Sen is the i-th word in the sentence Sen, and the function Idx(·) finds the position number of the specified word in the sentence Sen. For mask L The i-th column vector in , i∈[1,m] and is an integer, For mask r The j-th column vector in , j∈[1,m] is an integer, and “⊙” represents element-by-element multiplication;
[0084] S1.6 respectively in the left clause semantic representation C left and the right clause semantics C right A multi-head self-attention encoding process is performed on it to obtain the semantic encoding C′ of the left clause left and the semantic encoding of the right clause C′ right , the calculation process is as follows:
[0085]
[0086] Among them, MHSA(X) represents the input Multi-head attention MHA(Q,K,V);
[0087] S1.7 Semantically encode the left clause C′ left and the semantic encoding of the right clause C′ right Perform average pooling operation to obtain the left clause sentiment vector Z L and the right clause sentiment vector Z r , the calculation process is as follows:
[0088]
[0089] Among them, avePooling(C) represents the parameter Perform a column-wise average pooling operation;
[0090] S1.8 respectively for the left clause sentiment vector Z L and the right clause sentiment vector Z r Perform a linear transformation of softmax, calculate the probability of sentiment polarity, and obtain the final sentiment polarity. The calculation process is as follows:
[0091] u L =Q·Z L +b2 (18)
[0092]
[0093] u r =Q·Z r +b2 (21)
[0094]
[0095] in, is the representation matrix of sentiment polarity, is a bias vector, d k is the number of sentiment polarity categories, Y is the set of sentiment polarity categories, y is a sentiment polarity, Z L and Z r The corresponding sentiment polarity confidence score vector, P(y|Z L ,θ)、P(y|Z r ,θ) represent Z L and Z r The predicted probability of sentiment polarity y, y L ,y r They are the left and right emotional polarities of the final assessment, They represent the return value such that P(y|Z L ,θ) and P(y|Z r ,θ) is the maximum sentiment polarity, θ is the set of all learnable parameters, and exp(·) represents the exponential function with e as the base.
[0096] Furthermore, in step S1, a joint training method for jointly training a neural network model with automatic recognition of sentence context delimiters and automatic classification of dual-target entity sentiment polarities is:
[0097] (1) The cross entropy loss error is used to calculate the loss function of context segmentation symbol recognition and the loss function of dual-target entity sentiment analysis. The calculation process is as follows:
[0098]
[0099] Among them, Ω is the set of training sentences for the dual-target entity sentiment analysis task, |Ω| represents the size of the set Ω, is the word label of the context separator of the i-th training sentence in Ω, is the intermediate semantic representation of the i-th training sentence in Ω, are the left sentiment polarity label and the right sentiment polarity label of the i-th training sentence in Ω, are the left clause sentiment vector and the right clause sentiment vector of the i-th training sentence in Ω, respectively, Mid (θ) is the loss function used in contextual delimiter recognition training, Ψ L (θ) is the loss function used in the left target entity sentiment analysis training, Ψ r (θ) is the loss function used when training the right target entity sentiment analysis;
[0100] (2) Use the following formula (27) to calculate the joint loss function for joint training of sentence context segmentation symbol recognition and dual-target entity sentiment polarity classification:
[0101]
[0102] Among them, α1 and α2 are two weight parameters;
[0103] (3) The joint training goal is to minimize the joint loss error calculated by formula (27).
[0104] Application Examples
[0105] 1. Instance environment
[0106] This example uses the BERT-BASE version proposed and developed by Google AI Language in the document "Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Proceedings of the 2019 Conference of NAACL, pp 4171–4186" as the pre-training model of the BERT encoding layer. The BERT model includes 12 layers of Transformers, 768 hidden units, 12 multi-heads, and a total of 110M parameters); the multi-head attention used in this example comes from the document "Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention Is All You Need. In: 31st Conference on Neural Information Processing Systems (NIPS 2017), pp 5998–6008”, the number of attention heads is set to 12; to minimize the loss value, this example uses the Adam Optimizer optimizer, sets the learning rate to 2e-5, and the batch size to 16; during training, this example sets the epochs to 10.
[0107] 2. Dataset
[0108] This example uses the internationally widely used SemEval-2014task 4 dataset as the evaluation benchmark. The dataset was published at the 8th International Workshop on Semantic Evaluation in 2014. It provides two review datasets from the restaurant (Rest) and laptop (Lap) fields. Each sample in the SemEval-2014task 4 dataset consists of a review sentence, some opinion targets, and the corresponding sentiment polarity of the opinion targets. The details of the dataset are shown in Table 1.
[0109] Table 1 Dataset details
[0110]
[0111] 3. Comparison method
[0112] This example compares the model of the present invention with 5 non-BERT methods and 4 BERT-based methods. The comparison method is as follows:
[0113] (1) Non-BERT methods
[0114] MenNet[1] uses a multi-layer memory network combined with attention to capture the contribution of each context word to aspect polarity classification.
[0115] IAN[2] uses two LSTM networks to obtain the features of specific aspects and contexts respectively, then interactively generates their attention vectors, and finally connects the two attention vectors for aspect polarity classification.
[0116] TNet-LF[3] uses a CNN network to extract important features from word representations based on a bidirectional LSTM network.
[0117] A correlation-based mechanism is proposed to generate specific target representations of words in a sentence. The model also adopts position decay technology.
[0118] MCRF-SA[4] proposed a concise attention model based on multiple CRFs, which can extract specific
[0119] The model also adopts position decay and aspect connection techniques.
[0120] MAN[5] built two attention functions with position functions on top of the multi-layer transformer encoder: one for
[0121] Generate interactive attention between context and aspect relations, and a transformer-encoder based local attention from aspect to context.
[0122] (2) BERT-based methods
[0123] BERT-BASE[6] is a version of BERTBASE developed by Google AI Language, which uses a single sentence input method:
[0124] “[CLS]+comment sentence+[SEP]” is used for aspect polarity classification.
[0125] BERT-SPC[7] is the application of the pre-trained BERT model to the sentence pair classification (SPC) task. The input of BERT-SPC applied to the aspect polarity classification task is: “[CLS]+comment sentence+[SEP]+aspect target
[0126] +[SEP]”.
[0127] AEN-BERT[7] builds two multi-head attention mechanisms on top of the BERT encoder: a multi-head self-attention
[0128] A force mechanism is used to model the context, and an aspect-to-context multi-head attention mechanism is used to model the aspect target.
[0129] MAN-BERT is a variant of the MAN[5] model. This example uses the BERT model to replace the Transformer encoder in MAN[5].
[0130] The above-mentioned related documents are:
[0131] [1]Tang D, Qin B, Liu T (2016) Aspect Level Sentiment Classification with Deep Memory Network. In: Empirical methods in natural language processing, pp214–224
[0132] [2]Ma D, Li S, Zhang
[0133] [3]Li X,Bing L,Lam W,Shi B(2018)Transformation Networks for Target-Oriented Sentiment Classification.In Proceedings of ACL,pp 946-956
[0134] [4]Xu L,Bing L,Lu W,Huang F(2020)Aspect Sentiment Classification withAspect-Specific OpinionSpans.In Proceedings of EMNLP 2020,pp 3561-3567
[0135] [5]Xu Q, Zhu Li, Dai T, Yan C (2020) Aspect-based sentiment classification with multi-attention network. Neurocomputing, 388(3):135-143
[0136] [6]Devlin J,Chang MW,Lee K,Toutanova K(2019)BERT: Pre-training ofDeepBidirectional Transformers for Language Understanding.In:Proceedings ofthe
[0137] 2019Conference of NAACL,pp 4171–4186
[0138] [7]Song Y,Wang J,Jiang T,Liu Z,Rao Y(2019)Attentional encoder network for targeted sentence classification.In:arXiv preprint arXiv:1902.09314
[0139] 4. Example comparison results
[0140] This example evaluates various models by reporting the Accuracy(Acc) and Macro-average F1(M-F1) on the dataset.
[0141] Table 2 Experimental results. The ones with the symbol “+” are from the original paper, the ones with the symbol “*” are from the literature [5], and the others are from our experiments. The bold values represent the best
[0142]
[0143] The experimental results in Table 2 show that the multi-task learning dual-target entity sentiment analysis method proposed in the present invention achieves the best accuracy (Acc) and macro-average F1 (M-F1) results in the notebook and restaurant datasets, significantly exceeding the results of all similar methods, which fully demonstrates that the method of the present invention is feasible and excellent.
[0144] 5. Examples
[0145] For the sentiment sentence "Prices are higher todine but their food is quite good" containing two target entities "Prices" and "food", this example model first recognizes the context separator as "but", then obtains the semantic representation of the left clause "Prices are higher to dine" and the semantics of the right clause "their food is quitegood", and finally performs sentiment analysis on the semantics of the left clause and the right clause respectively, and obtains the sentiment polarity of the left target entity "Prices" as "negative" and the sentiment polarity of the right target entity "food" as "positive".
Claims
1. A dual-objective entity sentiment analysis method based on multi-task learning, characterized in that The following steps are involved: S1. Through multi-task learning of sentence context delimiter recognition and left and right entity sentiment polarity classification, a neural network model with automatic sentence context delimiter recognition and dual-target entity sentiment polarity automatic classification is jointly trained; S2. Using the neural network model trained in step S1 to identify contextual delimiters in the sentiment sentence; S3. In the neural network model trained in step S1, the semantic representation of the sentiment sentence is separated according to the position corresponding to the context separator obtained in step S2 to obtain the left clause semantic representation and the right clause semantic representation, and then sentiment analysis is performed on the left clause semantic representation and the right clause semantic representation respectively, and finally the sentiment polarity of the dual target entities is obtained; The emotional sentence refers to a multi-emotional expression sentence containing two target entities, left and right; The context separator refers to a word located between the left and right target entities in the emotional sentence, which separates the emotional expressions of the two target entities from each other; The neural network model refers to a neural network structure based on the BERT language model; The step S1 specifically includes: S1.1 The input sequence s of the BERT language model is composed of sentiment sentences Sen = {…, t1, w1, w2,…, w n ,t2,…} and BERT encoding symbols, as shown below: Mid={w1,w2,...,w n } (2) Where [CLS] is the encoding of the BERT classifier, [SEP] is the encoding of the BERT terminator, t1 is the left target entity to be analyzed, t2 is the right target entity to be analyzed, Mid = {w1, w2, ..., w n } is the middle word sequence between the left and right target entities t1 and t2, "..." represents the omitted word sequence, m is the length of the input sequence s, d w is the dimension of character encoding in BERT, n is the length of the middle word sequence Mid, and the "word" refers to the language segment separated by the Bert tokenizer Tokenzier; S1.2 Send the input sequence s to the BERT language model for processing to obtain the sentence semantic representation C of the emotional sentence Sen Sen , as shown below: Among them, BERT(·) represents the BERT language model, is the i-th hidden state of the BERT language model, d b is the number of hidden units of the BERT language model; S1.3 According to the corresponding relationship, from C Sen Extract the middle word sequence Mid={w1,w2,...,w n The intermediate semantics corresponding to C Mid , as shown below: Among them, Middle(·) represents intermediate semantic extraction, is the i-th middle word w i In C Sen The corresponding hidden state in ; S1.4 Intermediate semantic representation C Mid Perform a softmax linear transformation to identify contextual delimiters. The calculation process is as follows: γ=C Mid G+b1 (5) in, is a learnable parameter vector for contextual delimiter recognition, b1∈R is a bias parameter, "·" represents the dot product operation of the vector, and γ∈R n is the context segmentation confidence score vector corresponding to the middle word sequence Mid, w is an middle word, P(w|C Mid ,θ) represents the predicted probability that the middle word w is a context separator, It means to return the value that makes P(w|C Mid ,θ) is the middle word with the maximum value, w * is the calculated contextual delimiter, θ is the set of all learnable parameters, and exp(·) represents the exponential function with e as the base; S1.5 in w sp As a separator, two mask matrices consisting of 1 and 0 are formed to represent the sentence semantics C Sen Separated into left clause semantic representation C left and the right clause semantics C right , the calculation process is as follows: Among them, mask L is the mask matrix used to separate the semantics of the left clause, mask r is the mask matrix used to separate the semantics of the right clause, is a vector of all 1s, is a vector of all zeros, tonken i ∈Sen is the i-th word in the sentence Sen, and the function Idx(·) finds the position number of the specified word in the sentence Sen. For mask L The i-th column vector in , i∈[1,m] and is an integer, For mask r The j-th column vector in , j∈[1,m] is an integer, "⊙" means element-by-element multiplication; S1.6 respectively in the left clause semantic representation C left and the right clause semantics C right A multi-head self-attention encoding process is performed on it to obtain the semantic encoding C′ of the left clause left and the semantic encoding of the right clause C′ right , the calculation process is as follows: Among them, MHSA(X) represents the input Multi-head attention MHA(Q,K,V); S1.7 Semantically encode the left clause C′ left and the semantic encoding of the right clause C′ right Perform average pooling operation to obtain the left clause sentiment vector Z L and the right clause sentiment vector Z r , the calculation process is as follows: Among them, avePooling(C) represents the parameter Perform a column-wise average pooling operation; S1.8 respectively for the left clause sentiment vector Z L and the right clause sentiment vector Z r Perform a linear transformation of softmax, calculate the probability of sentiment polarity, and obtain the final sentiment polarity. The calculation process is as follows: u L =Q·Z L +b2 (18) u r =Q·Z r +b2 (21) in, is the representation matrix of sentiment polarity, is a bias vector, d k is the number of sentiment polarity categories, Y is the set of sentiment polarity categories, y is a sentiment polarity, Z L and Z r The corresponding sentiment polarity confidence score vector, P(y|Z L ,θ)、P(y|Z r ,θ) represent Z L and Z r The predicted probability of sentiment polarity y, y L ,y r They are the left and right emotional polarities of the final assessment, They represent the return value such that P(y|Z L ,θ) and P(y|Z r ,θ) is the maximum sentiment polarity, θ is the set of all learnable parameters, and exp(·) represents the exponential function with e as the base; In step S1, a joint training method for jointly training a neural network model with automatic recognition of sentence context delimiters and automatic classification of dual-target entity sentiment polarities is: (1) The cross entropy loss error is used to calculate the loss function of context segmentation symbol recognition and the loss function of dual-target entity sentiment analysis. The calculation process is as follows: Among them, Ω is the set of training sentences for the dual-target entity sentiment analysis task, |Ω| represents the size of the set Ω, is the word label of the context separator of the i-th training sentence in Ω, is the intermediate semantic representation of the i-th training sentence in Ω, are the left sentiment polarity label and the right sentiment polarity label of the i-th training sentence in Ω, are the left clause sentiment vector and the right clause sentiment vector of the i-th training sentence in Ω, respectively, Mid (θ) is the loss function used in contextual delimiter recognition training, Ψ L (θ) is the loss function used in the left target entity sentiment analysis training, Ψ r (θ) is the loss function used when training the right target entity sentiment analysis; (2) Use the following formula (27) to calculate the joint loss function for joint training of sentence context segmentation symbol recognition and dual-target entity sentiment polarity classification: Among them, α1 and α2 are two weight parameters; (3) The joint training goal is to minimize the joint loss error calculated by formula (27).
