A multi-task learning-based aspect sentiment analysis model

By combining the multi-task learning model with aspect category detection and aspect level sentiment analysis modules, the problem of unsatisfactory multi-task processing effect in the prior art is solved, and more accurate aspect level sentiment analysis is achieved.

CN114722142BActive Publication Date: 2024-11-19DALIAN NATIONALITIES UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210108256.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-28
Publication Date
2024-11-19
Estimated Expiration
2042-01-28

AI Technical Summary

Technical Problem

When the existing sentiment analysis model processes multiple subtasks at the same time, the effect is not ideal, and the information correlation between subtasks cannot be effectively captured, resulting in a degradation of model performance.

Method used

A aspect sentiment analysis model based on multi-task learning is proposed, combining aspect category detection module and aspect level sentiment analysis module, and sharing information through ACD embedding layer, bidirectional long and short-term memory network layer, self-attention mechanism layer and aspect category prediction layer to improve model performance.

Benefits of technology

Through the multi-task learning model, the performance of aspect-level sentiment analysis has been successfully improved, and the semantic information between aspect words and aspect categories can be effectively captured, improving the accuracy and effectiveness of sentiment analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114722142B_ABST
    Figure CN114722142B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of natural language processing, and relates to an aspect sentiment analysis model based on multi-task learning. It includes: an aspect category detection module and an aspect-level sentiment analysis module; the two parts are aggregated to obtain the final aspect category sentiment analysis result; the aspect category detection module includes an ACD embedding layer for processing data, a bidirectional long short-term memory network layer, a self-attention mechanism layer, and an aspect category prediction layer in sequence; the aspect-level sentiment analysis module includes an ACSA embedding layer for processing data, a multi-layer Bi‑LSTM, a word sentiment prediction layer, and an aspect category sentiment prediction layer in sequence. Beneficial effect: The present invention proposes a neural network model that introduces an auxiliary task aspect category detection ACD to perform ACSA tasks. Combining the advantages of the two models and on public data sets, good performance is obtained. Multi-task learning has been successfully applied to deep learning tasks by utilizing the commonalities and differences between tasks to improve performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of natural language processing and relates to an aspect sentiment analysis model based on multi-task learning. Background Art

[0002] Aspect-level sentiment analysis includes different subtasks. In general research, the focus is on one of the subtasks. When only one task is studied, the connection between the subtasks cannot be satisfied at the same time. This problem can be solved by using a series connection method, but this will lose the end-to-end training data, resulting in unsatisfactory experimental results. Due to the characteristics of aspect-level sentiment analysis, there is a certain information correlation between aspect words or aspect category extraction and aspect-level sentiment analysis. As a research focus in the field of natural language processing, aspect-level sentiment analysis using multi-task learning methods has attracted more and more attention.

[0003] For aspect-level sentiment analysis, the current mainstream research direction of multi-task learning is roughly divided into three parts: the first is to complete the aspect word extraction and aspect category extraction tasks at the same time, for example, in a text, it is to capture the semantic information between aspect words and aspect categories; the second is to complete the aspect word extraction and sentiment analysis of aspect words at the same time; the third is to complete the aspect category extraction and aspect category sentiment analysis at the same time. The second and third tasks are to establish a joint mechanism to avoid the unsatisfactory model effect caused by the errors of the serial method, and at the same time, it can also capture relevant semantic information. Summary of the invention

[0004] In order to overcome some shortcomings and deficiencies of the existing aspect sentiment analysis model, the present invention proposes the following technical solution: an aspect sentiment analysis model based on multi-task learning, comprising: an aspect category detection module and an aspect level sentiment analysis module; the two parts are aggregated to obtain the final aspect category sentiment analysis result;

[0005] The aspect category detection module includes an ACD embedding layer for processing data, a bidirectional long short-term memory network layer, a self-attention mechanism layer, and an aspect category prediction layer in sequence;

[0006] The aspect-level sentiment analysis module includes, in sequence, an aspect category sentiment analysis embedding layer for processing data, a multi-layer Bi-LSTM, a word sentiment prediction layer, and an aspect category sentiment prediction layer.

[0007] Furthermore, in the ACD embedding layer, words are converted into vectors; the sentence S that has undergone data preprocessing is segmented into word sequences, and then the vector representation of the sentence is obtained through the embedding matrix Ww, and the sentence is converted into a vector sequence.

[0008] Furthermore, the Bi-LSTM layer includes forward propagation and backward propagation, respectively learning different points of context information, and finally connecting the hidden states to the output layer for output at the same time.

[0009] Furthermore, the aspect category prediction layer uses the weights of the self-attention mechanism and the output of the bidirectional LSTM as the input of the prediction layer. The model uses the weighted hidden state as the sentence representation of the ACD prediction, for the jth aspect category, as shown in formulas 3.13 and 3.14:

[0010]

[0011]

[0012] Where W j ∈R D×1 and b j is a scalar; is the sentence representation of the j-th aspect category and self-attention; H is the hidden representation through Bi-LSTM.

[0013] Furthermore, the word sentiment prediction layer uses the hidden state of the l-th layer Bi-LSTM at the i-th step As the representation of the i-th word, two fully connected layers are used to produce the sentiment prediction p of the i-th word i As shown in formula 3.19:

[0014]

[0015] Where W 1 ∈R d×d , W 2 ∈R d×3 , b 1 ∈R d , b 2 ∈R 3 is a learnable parameter; is the hidden state output by the l-th Bi-LSTM at each time step i.

[0016] Furthermore, the weights provided by the ACD embedding layer aggregate the sentiment predictions of the words to obtain the sentiment predictions of the aspect categories. For the jth aspect category, its sentiment p j It can be expressed by formula 3.20:

[0017]

[0018] Among them, p j ∈R 3 , Represents the weight vector α provided by the ACD task jThe weight of the i-th word in the j-th aspect category; i is the sentiment prediction of the i-th word.

[0019] For the model loss function setting, in the aspect category detection task, the loss function is defined as shown in formula 3.21

[0020]

[0021] Among them, yj represents the true label, is the predicted label.

[0022] In the aspect-level sentiment analysis task, the loss function is defined as shown in Formula 3.22:

[0023]

[0024] Among them, yj c For the case where the sample is of the jth aspect category and the cth sentiment polarity, pj c This is the case where the predicted sample is of the jth aspect category and has the cth sentiment polarity.

[0025] The loss function obtained by jointly training the model to complete the two tasks is shown in Formula 3.23:

[0026]

[0027] Among them, β is the weight of ACSA loss, λ is the L2 regularization factor, and θ is the parameter set of the model.

[0028] Beneficial effects: This paper proposes a neural network model that introduces the auxiliary task Aspect Category Detection (ACD) to perform ACSA tasks. Combining the advantages of both models, and on public datasets, good results are obtained. By utilizing the commonalities and differences between tasks to improve performance, multi-task learning has been successfully applied to deep learning tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Structure diagram of aspect-level sentiment analysis network model based on multi-task learning;

[0030] Figure 2 ACD classifier model structure diagram;

[0031] Figure 3 Aspect word sentiment representation structure diagram;

[0032] Figure 4 Aggregation structure diagram. DETAILED DESCRIPTION

[0033] 1.1 Basic Framework of Aspect-Level Sentiment Analysis with Multi-Task Learning

[0034] 1.1.1 Model Problem Assumptions

[0035] The purpose of aspect-level sentiment analysis is to find the aspect information of the evaluation text and its corresponding sentiment polarity. The aspect information can be extracted from the comment text or can be a predefined aspect category. Therefore, aspect-level sentiment analysis can be divided into tasks such as aspect term extraction (ATE), aspect category detection (ACD), aspect term sentiment analysis (ATSA), and aspect category sentiment analysis (ACSA). Aspect term extraction (ATE) refers to extracting all aspect terms contained in unstructured text; aspect category detection (ACD) is to find out the aspect categories of all opinion evaluations contained in the comment text based on a set of predefined aspect categories; aspect term sentiment analysis (ATSA) and aspect category sentiment analysis (ACSA) are to determine the sentiment polarity information expressed by the comment text for aspect terms and aspect categories l respectively. Aspect term sentiment analysis (ATSA) and aspect category sentiment analysis (ACSA) are collectively referred to as aspect-level sentiment analysis (ABSA).

[0036] In the Aspect Category Sentiment Analysis task (ACSA), assume that there are N predefined aspect category labels A = {a1, a2, …, a N} and predefined sentiment polarity P = {Neg, Neu, Pos}. Given a sentence, S = {w1,w2,…,w n} and K aspect categories, As mentioned in S, the ACSA task predicts the sentiment polarity distribution of K aspect categories, p = {p1, p2, ..., p K},in Multi-instance multi-label learning assumes that for the k-th aspect category, p k is an unknown function of the unobserved word-level sentiment distribution. Our model first generates a sentiment distribution p for each word. j , which are then combined into a sentence-level prediction.

[0037] Specifically, as shown in formulas (3.1) and (3.2):

[0038]

[0039]

[0040] 1.1.2 Model Framework

[0041] At present, most of the aspect-level sentiment analysis tasks are implemented based on a single-task network structure. This section proposes an aspect-level sentiment analysis algorithm based on multi-task learning. The multi-task structure is used to jointly learn the two subtasks of aspect-level sentiment analysis, so that tasks can share information and improve the performance of tasks, thus solving the problem of poor performance of single-task network structure. The neural network model of aspect-level sentiment analysis based on multi-task learning is as follows: Figure 1 As shown in the figure, the model is mainly divided into two modules: aspect category detection module (ACD) and aspect level sentiment analysis module (ACSA).

[0042] Specifically, the paper model is mainly divided into two parts, one is the self-attention-based aspect category detection classifier (ACD), and the other is the aspect category sentiment analysis classifier (ACSA). ACD is used as an auxiliary task to generate word weights for each aspect category, and the weights represent the probability of a word being an aspect category. ACSA first predicts the sentiment of the vocabulary, and then combines the corresponding weights and the sentiment of the vocabulary to obtain the final sentiment analysis of each aspect category.

[0043] The ACD part includes four modules: embedding layer, Bi-LSTM layer, self-attention mechanism layer and aspect category prediction layer, and the ACSA part also includes four modules: embedding layer, multi-layer Bi-LSTM, word sentiment prediction layer and aspect category sentiment prediction layer. In the ACD task, each aspect category shares the embedding layer and LSTM layer, and has different attention layers and aspect category prediction layers. In the ACSA task, each aspect category shares the embedding layer, multi-layer Bi-LSTM and word sentiment prediction layer, and has different aspect category sentiment prediction layers.

[0044] 2.1 Aspect Category Detection Method Based on Self-Attention Mechanism

[0045] In a review text, a user comment may cover different aspects of a product or an event, and show different sentiment tendencies towards these aspects. To better explore the user's attitude towards a product or an event and analyze the sentiment tendency expressed by the user in each aspect of the product or event, the primary task is to identify the aspects of the product involved in the user comment. However, in many products or events, users mostly implicitly express a certain aspect of the product. For example, in the sentence "While it was large and a bit noisy, the drinks were fantastic, and the food was superb.", this sentence involves both words like "drinks" and "food" that can explicitly express aspects, and words like "large" and "noisy" that implicitly express aspects. From this example, it can be seen that both "drinks" and "food" can be replaced by the aspect category "food", which explicitly indicates the aspect category of "food". However, "large" and "noisy" are not aspect words. They only express the sentiment towards the restaurant environment and implicitly express the aspect category "ambience". Therefore, in analyzing the user's attitude towards a product or an event, finding the aspect category is a very crucial step, which can improve the accuracy of aspect category sentiment analysis.

[0046] 2.1.1 Overall Process

[0047] The self-attention mechanism neural network can reduce the time complexity to a certain extent. In terms of long-distance dependencies, since the self-attention mechanism calculates the attention for each word, it can represent the important features of the data, obtain the dependency relationships in the sentence, acquire semantic features, and finally increase the computational parallelism. Therefore, using the self-attention mechanism and bidirectional LSTM for aspect category detection retains the advantages of the self-attention mechanism. Its overall process is as Figure 2 shown.

[0048] 2.2.1 Detailed Introduction

[0049] The detailed process of the aspect category detection experiment is as follows:

[0050] (1) Data preprocessing.

[0051] In the original text, there are usually some words that have no practical meaning, such as auxiliary words like "de" (的) and "le" (了), and it also contains some special symbols, such as "@" and "&". After research, it is found that these words and symbols with no practical meaning are meaningless for the research of aspect-level sentiment analysis and may even affect the result of sentiment analysis. Therefore, deleting these elements can simplify the content of the text, reduce the influence of noise, and improve the accuracy of sentiment analysis.

[0052] When the processed text is saved in a file, the format of each piece of data is as shown in formula (3.3):

[0053] S={w1,w2,…,w n} (3.3)

[0054] Where n is the length of the sentence, w n For a single word in a sentence.

[0055] (2) Embedding layer

[0056] The input of natural language processing is a piece of text, that is, a one-dimensional word sequence. The value of the text is discrete and is a kind of unstructured data. Therefore, natural language processing must first represent the text as numbers or vectors that can be directly calculated by computers. At present, the text representation in natural language processing mainly adopts word embedding technology. This article mainly uses two word embedding methods, Glove and BERT.

[0057] The embedding layer converts words into vectors to map words from semantic space to vector space. In the input stage of the model, it is a sentence consisting of n words, S = {w1,w2,…w n}, firstly, the sentence S is segmented into word sequences, and then the embedding matrix W is used w To obtain the vector representation of the sentence, convert the sentence into a vector sequence. Its specific vector representation is shown in formula (3.4):

[0058]

[0059] Among them, X D is a vector sequence, W w ∈R d×|V| , d is the dimension of the embedded word, and |V| is the size of the vocabulary.

[0060] (3) Bidirectional LSTM layer

[0061] In order to simultaneously consider the contextual semantic information of the text and fully extract all the features contained in the text, the output result after the text is represented by vector embedding is used as the input of the bidirectional LSTM. The bidirectional LSTM consists of two parts, forward propagation and backward propagation. The learned contextual information is different, and finally the hidden state is connected to the output layer for output. Combining contextual information with word embedding is an effective way to express words. LSTM has achieved great success in various sentiment analysis tasks because it can effectively capture the sequence information of input sentences. In these works, sentiment analysis requires the context of keywords to infer their hidden sentiment information. However, LSTM can only process sequences from one direction. In order to capture the forward and backward sequence information of the input sequence, we perform forward LSTM encoding on each word and obtain the sequence information representation from left to right. Use backward LSTM from right to left to obtain sequence information representation Then, we connect the two vector representations in opposite directions to generate the final sequence representation. The specific formulas are shown in (3.5), (3.6), and (3.7):

[0062]

[0063]

[0064]

[0065] hi is the final sequence representation.

[0066] (4) Self-Attention Layer

[0067] To extract the global dependencies in the input sentence, the problem that needs to be solved is to calculate the attention of each word in the sentence. In order to capture the overall dependencies of the words in the sentence, the first step is to calculate the attention of each word in the sentence and query the set of all words in the sentence. In the field of natural language processing, keys and values ​​are often referred to as input sequences, which means query = key = value. Self-attention is a scalable and parallel attention calculation method, which is usually considered as a content-based query processing. It calculates the attention function of a set of queries and packs them into a matrix Q. At the same time, keys and values ​​are also packed into K and V.

[0068] In order to get i (i∈[1,n]) and s j (j∈[1,n]) in S, we need to calculate s i With s j Then, s i and j Packed into matrices Q and K respectively. TThe matrix obtained by multiplication represents the relationship between each word and the entire sentence. The relevance score of each word is generated by the softmax function. Finally, this score is multiplied by the mapping matrix V of S to obtain the word representation that captures the global dependency information.

[0069] The way to utilize aspect information in the self-attention mechanism is to let aspect embedding play a role in the calculation of attention weights. The model adds the input aspect information to each word vector, as shown in formula (3.8):

[0070]

[0071] Among them, H t ={h1,h2,…,h m} is the hidden representation through Bi-LSTM. FFNN is a feed-forward neural network.

[0072] Then Q, K, and V are the mapping matrices of the input sentence, which can be initialized by multiplying the embedding of the input sentence with the corresponding weight matrix, as shown in (3.9), (3.10), and (3.11):

[0073]

[0074]

[0075]

[0076] Among them, Q,K,V∈R n×2dim is a mapping of word representations with aspect information. Calculate the attention score and obtain the sentence representation Z of self-attention, as shown in (3.12):

[0077]

[0078] in, is the scaling factor, set to the dimensionality of the latent representation.

[0079] (5) Aspect Category Prediction Layer

[0080] In the output layer of aspect category prediction, the weight of the self-attention mechanism and the output of the bidirectional LSTM are used as the input of the prediction layer, and the model uses the weighted hidden state as the sentence representation of ACD prediction. For the jth aspect category. Its formula is shown in (3.13) and (3.14):

[0081]

[0082]

[0083] Where W j ∈RD×1 are learnable parameters and b j is a scalar.

[0084] 3.1 Aspect Word Sentiment Analysis Method Based on Multi-layer Bidirectional LSTM

[0085] In the aspect-level sentiment analysis task, a review text needs to consider both sentence and aspect information to make judgments on sentiment polarity. At present, the research on aspect-level sentiment analysis is constantly deepening. Because neural networks can automatically encode raw information into continuous low-dimensional vectors, avoiding complex feature engineering, more and more researchers are paying attention to neural network-based methods.

[0086] In a review text, since traditional LSTM neural networks may ignore some contextual dependency information, the bidirectional LSTM can take into account the beginning and end information of the text so as to learn implicit contextual dependency information. In addition, the multi-layer bidirectional long short-term memory network is used to extract information at different levels for aspect-level sentiment analysis, which can retain the advantages of the original long-short distance dependency and include the advantages of multi-layer LSTM.

[0087] 3.1.1 Overall Process

[0088] This section mainly uses a multi-layer bidirectional LSTM neural network to perform aspect-level sentiment analysis. In the model, the comment text is first represented by a vector, and a multi-layer Bi-LSTM network is used to extract the text context relationship features, and then the sentiment representation of the words in the comment text is obtained. Its structure is as follows Figure 3 shown.

[0089] 3.2.1 Detailed Introduction

[0090] (1) Embedding layer

[0091] For the embedding layer of the aspect word sentiment representation module, its specific data is the same as the aspect category detection embedding layer, and the embedding method is also Glove and BERT word embedding. The data embedding form is the same. This embedding layer also uses the embedding matrix to convert sentences into vectors. The specific expression is shown in formula (3.15):

[0092]

[0093] (2) Multi-layer Bi-LSTM layer

[0094] In natural language processing tasks, most practices are to use language models to perform word embedding operations first, and then continue with subsequent downstream tasks. In existing research, most of the methods used are vector addition methods, but most of these methods ignore the constraints and restrictions on the order of sentences, resulting in incorrect feature extraction and a decrease in the final training effect. Using LSTM can capture the problem of long-distance dependencies. Through training, LSTM can learn what information and forget what information. In order to better capture bidirectional semantic information, the hidden output of bidirectional LSTM is used. When using multi-layer bidirectional LSTM, it is mainly to increase the number of layers of bidirectional LSTM. The feature vector value is the output result of the last layer. In the experiment, after increasing the number of Bi-LSTM layers, the results are significantly improved.

[0095] The word vector output by the embedding layer is fed into the multi-layer bidirectional LSTM as the input of the multi-layer Bi-LSTM. Each layer takes the output of the previous layer as input. Formally speaking, the hidden state of the (l-1)th layer is: The lth Bi-LSTM outputs the hidden state:

[0096] At each time step i, the hidden state The calculation method of is shown in (3.16), (3.17) and (3.18):

[0097]

[0098]

[0099]

[0100] in, represents the size of the LSTM hidden state. The total number of BiLSTM layers is L.

[0101] (3) Word sentiment prediction layer

[0102] The word sentiment prediction layer is the last layer of the model structure. Since this experiment is to obtain the final aspect category sentiment analysis results, in order to reduce the difficulty of training, only the sentiment representation of the word is obtained in the model, and softmax is not used for output. If softmax is used for output, a lot of sentiment information will be lost, resulting in a decrease in the performance of the final aspect category sentiment analysis results.

[0103] In the word sentiment prediction layer, the hidden state of the l-th layer Bi-LSTM at the i-th step is used As the representation of the i-th word, two fully connected layers are used to produce the sentiment prediction p of the i-th word i As shown in (3.19):

[0104]

[0105] Where W 1 ∈R d×d , W 2 ∈R d×3 , b 1 ∈R d , b 2 ∈R 3 are learnable parameters.

[0106] 4.1 Aggregation Method Based on Multi-Task Learning

[0107] Aspect Category Sentiment Analysis (ACSA) is a very important subtask in aspect-level sentiment analysis. This task obtains the sentiment polarity of the aspect category to which the target word in the sentence belongs based on the given sentence and predefined aspect categories. In the sentence "The food is good, but the decoration is not good.", the aspect word "food" belongs to the "food" aspect category, and its sentiment is positive, while the aspect word "decoration" belongs to the "ambience" aspect category, and its sentiment is negative. The main purpose of aspect category sentiment analysis is to detect the sentiment of aspect categories in the review text. The previous two parts respectively introduced the aspect category detection method based on Bi-LSTM and self-attention mechanism and the aspect-level sentiment analysis method based on multi-layer Bi-LSTM. In this part, the two parts are mainly aggregated to obtain the final aspect category sentiment analysis result. Its specific form is as follows: Figure 4 shown.

[0108] The model aggregates the sentiment predictions of words according to the weights provided by the self-attention layer of the ACD task classifier to obtain the sentiment predictions of aspect categories. For the jth aspect category, its sentiment p j It can be expressed by formula (3.20):

[0109]

[0110] Among them, p j ∈R 3 , p i represents the sentiment prediction of the i-th word, Represents the weight vector α provided by the ACD task j The weight of the i-th word in with respect to the j-th aspect category.

[0111] For the model loss function setting, in the aspect category detection task, since each prediction is a binary classification problem, the loss function is defined as shown in formula (3.21)

[0112]

[0113] y j represents the true label, is the predicted label.

[0114] In the aspect-level sentiment analysis task, because the loss of the K aspect categories mentioned in the sentence is included, the loss function is defined as shown in formula (3.22):

[0115]

[0116] y jc is the case where the sample is of the jth aspect category and has the cth sentiment polarity, and pjc is the case where the predicted sample is of the jth aspect category and has the cth sentiment polarity.

[0117] The loss function obtained by jointly training the model to complete the two tasks is shown in formula (3.23):

[0118]

[0119] Among them, β is the weight of ACSA loss, λ is the L2 regularization factor, and θ contains all the parameters of our model.

[0120] 5.1 Summary

[0121] In the aspect-level sentiment analysis task, a single-task model cannot share its own features, or the end-to-end training data will be lost when using a series method, resulting in unsatisfactory model training results. Therefore, aspect category sentiment analysis based on multi-task learning is proposed. The model is mainly divided into two parts. The weight parameters of aspect category detection are shared by auxiliary tasks to promote multi-task joint learning. The aspect-level sentiment analysis model based on multi-task learning is introduced, which mainly includes three parts: the overall framework, the aspect category detection subtask model framework, and the aspect-level sentiment analysis model framework. Through the specific introduction of each part of the model, we have a deeper understanding of the use of multi-task learning for aspect-level sentiment analysis. Through the introduction of each part of the framework, the final aspect-level sentiment analysis result is achieved.

Claims

1. A multi-task learning-based aspect sentiment analysis model, characterized in that: include: Aspect category detection module and aspect-level sentiment analysis module; Aggregate the two parts to get the final aspect category sentiment analysis result; The aspect category detection module includes an ACD embedding layer for processing data, a bidirectional long short-term memory network layer, a self-attention mechanism layer, and an aspect category prediction layer in sequence; The aspect-level sentiment analysis module includes, in sequence, an aspect-category sentiment analysis embedding layer for processing data, a multi-layer Bi-LSTM, a word sentiment prediction layer, and an aspect-category sentiment prediction layer; The aspect category prediction layer uses the weights of the self-attention mechanism and the output of the bidirectional long short-term memory network as the input of the prediction layer. The model uses the weighted hidden state as the sentence representation of the aspect category detection prediction, for the jth aspect category, as shown in formulas 3.13 and 3.14: Where W j ∈R D×1 and b j is a scalar, is the sentence representation of the j-th aspect category and self-attention; H is the hidden representation through the bidirectional long short-term memory network; The word sentiment prediction layer uses the hidden state of the l-th layer Bi-LSTM at the i-th step As the representation of the i-th word, two fully connected layers are used to produce the sentiment prediction p of the i-th word i As shown in formula 3.19: Where W 1 ∈R d×d , W 2 ∈R d×3 , b 1 ∈R d , b 2 ∈R 3 is a learnable parameter, For each time step i, the l-th Bi-LSTM outputs the hidden state; The weights provided by the ACD embedding layer aggregate the sentiment predictions of the words to obtain the sentiment predictions of the aspect categories. For the jth aspect category, its sentiment p j It is expressed by formula 3.20: Among them, p j ∈R 3 , Represents the weight vector α provided by the aspect category detection task j The weight of the i-th word in the j-th aspect category; i is the sentiment prediction of the i-th word; For the model loss function setting, in the aspect category detection task, the loss function is defined as shown in formula 3.21 Among them, y j represents the true label, is the predicted label; In the aspect-level sentiment analysis task, the loss function is defined as shown in Formula 3.22: Among them, yj c For the case where the sample is of the jth aspect category and the cth sentiment polarity, pj c For the case where the predicted sample is of the jth aspect category and has the cth sentiment polarity; The loss function obtained by jointly training the model to complete the two tasks is shown in Formula 3.23: Among them, β is the weight of the loss value in the aspect-level sentiment analysis task, λ is the L2 regularization factor, and θ is the parameter set of the model.

2. The aspect sentiment analysis model based on multi-task learning according to claim 1, characterized in that: In the ACD embedding layer, words are converted into vectors; the sentence S that has been preprocessed by data is segmented into word sequences, and then the vector representation of the sentence is obtained through the embedding matrix Ww, and the sentence is converted into a vector sequence.

3. The aspect sentiment analysis model based on multi-task learning according to claim 1, characterized in that: The bidirectional long short-term memory network layer includes forward propagation and backward propagation, which respectively learn different points of context information, and finally connect the hidden states to the output layer for output.