Aspect-level sentiment quadruple extraction method and system based on deep learning

Abstract

The present invention belongs to the field of deep learning, and provides an aspect-level sentiment quadruple extraction method and system based on deep learning, which obtains an original given sentence; based on the original given sentence, uses a pre-trained aspect-level sentiment quadruple extraction model to extract sentiment quadruple; wherein, using the pre-trained aspect-level sentiment quadruple extraction model to extract sentiment quadruple includes: extracting aspect words in the original given sentence by using inline tags; constructing a question template sentence based on the extracted aspect words, and splicing the question template sentence and the original given sentence, and extracting opinion words paired with the aspect words based on the spliced sentence; using predefined aspect categories and predefined aspect sentiment polarities to generate corresponding aspect category decoder template sentences and sentiment polarity decoder template sentences, and decoding according to the encoded original given sentence to obtain aspect categories and aspect sentiments.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning technology, and specifically relates to a method and system for extracting aspect-level sentiment quadruple based on deep learning. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] In recent years, aspect-sentiment quadruple prediction (ASQP) has become a popular task in aspect-level sentiment analysis. Its goal is to decode a given sentence into an aspect-sentiment quadruple (aspect category, aspect word, opinion word, and sentiment polarity). Efficiently extracting aspect-sentiment quadruple often encounters the following problems: 1. The intrinsic relationship between aspect and opinion words is often ignored, resulting in a lack of correlation between aspect-opinion pairs and neglecting the mutual interference between different sentiment quadruples; 2. The semantic information of the sentiment elements in the comment sentence is often not fully utilized, increasing the risk of inaccurate predictions.

[0004] Cai et al. studied the aspect-sentiment quadruple prediction (ASQP) task, emphasizing implicit aspects or opinions. The authors argued that implicit aspects or opinions often appear in real-world scenarios and used "null" to represent them in sentiment quadruple. They introduced two new datasets annotated with sentiment quadruple annotations and constructed a series of pipeline baselines by combining existing models to benchmark the task. Subsequently, researchers have attempted to address the ABSA problem using a Seq2Seq approach by targeting the desired sequence of sentiment elements in generative models. Zhang et al. proposed a Paraphrase modeling strategy to predict sentiment quadruple in an end-to-end manner. By combining annotated sentiment elements with pre-established templates and using the obtained natural language sentences as target sequences, they transformed the original quadruple prediction task into a text generation problem and solved it using the Seq2Seq modeling paradigm. Subsequently, Hu et al. found that the order of sentiment elements in the template affects the performance of quadruple extraction and proposed that combining multiple templates can improve the ASQP task through data augmentation. However, although these models are able to exploit rich label semantics by encoding natural language labels into the target output, they cannot effectively capture the semantic structure between aspect words and opinion words.

[0005] Among these subtasks, aspect-level sentiment analysis can be divided into two categories: extraction tasks and classification tasks. In the extraction task, pairing aspect words and opinion words (i.e., the process of forming aspect-opinion pairs) is more difficult and often results in inaccuracies. Previous work has demonstrated that it is feasible to use generative models to extract aspect categories and sentiment polarity as a classification task. However, traditional classification tasks are performed by using neural networks on top of pre-trained representations, with separate network parameters. At the same time, the integration of aspect categories makes the input representation of specific aspects not entirely a natural language sentence, which is different from the pre-training setting. Intuitively, by connecting pre-training and classification tasks at the task level, rather than just at the representation level, more pre-training knowledge can be utilized. Summary of the Invention

[0006] In order to solve the above problems, the present invention proposes an aspect-level sentiment quadruple extraction method and system based on deep learning. The present invention adopts a two-stage framework in aspect-sentiment quadruple prediction, first extracting aspect-opinion pairs, and then classifying aspect categories and sentiment polarity.

[0007] According to some embodiments, a first solution of the present invention provides an aspect-level sentiment quadruple extraction method based on deep learning, which adopts the following technical solutions:

[0008] A deep learning-based aspect-level sentiment quadruple extraction method, including:

[0009] Get the original given statement;

[0010] Based on the original given sentence, sentiment quadruple extraction is performed using the pre-trained aspect-level sentiment quadruple extraction model;

[0011] Among them, the emotion quadruple extraction is performed using a pre-trained aspect-level emotion quadruple extraction model, including:

[0012] The aspect words in the original given sentence are extracted by using inline tags;

[0013] Constructing a question template sentence based on the extracted aspect words, and splicing the question template sentence with the original given sentence, and extracting opinion words paired with the aspect words based on the spliced sentence;

[0014] The predefined aspect categories and predefined aspect sentiment polarities are used to generate corresponding aspect category decoder template sentences and sentiment polarity decoder template sentences, and the encoded original given sentences are decoded to obtain aspect categories and aspect sentiments.

[0015] Furthermore, a sentiment quadruple is formed based on the extracted aspect words, opinion words, aspect categories and aspect sentiments.

[0016] Furthermore, the aspect words in the original given sentence are extracted using the inline tag method, specifically:

[0017] Based on the original given sentence, the BERT encoder is used to obtain the embedded semantic representation of the original given sentence;

[0018] Based on the embedded semantic representation, the flag with a predicted probability greater than a set value is regarded as the beginning or end of the aspect element;

[0019] Pair each beginning token with its nearest ending token to determine an aspect word;

[0020] By analogy, it is determined to extract all aspect words in the original given sentence.

[0021] Furthermore, a question template sentence is constructed based on the extracted aspect words, and the question template sentence is concatenated with the original given sentence, including:

[0022] Construct question template sentences using fixed template sentences based on aspect words;

[0023] Based on the question template statement, the original given statement is spliced;

[0024] Get the concatenated statement.

[0025] Furthermore, the extraction of opinion words paired with aspect words based on the concatenated sentences is specifically as follows:

[0026] Based on the concatenated sentences, the BERT encoder is used to obtain the opinion representation of the original given sentence;

[0027] Opinion representation based on the original given sentence, using the flag whose predicted probability is greater than the set value as the beginning or end of the opinion word;

[0028] Pair each start token with its nearest end token to determine an opinion word;

[0029] By analogy, it is determined to extract all opinion words in the opinion representation of the original given sentence.

[0030] Furthermore, the predefined aspect categories are used to generate corresponding aspect category decoder template sentences, and the encoded original given sentences are decoded to obtain the aspect categories corresponding to the aspect words, specifically:

[0031] Use the BART encoder to encode the original given sentence to obtain the hidden representation of the original given sentence;

[0032] Generate corresponding aspect category decoder template sentences using predefined aspect categories;

[0033] The decoder's attention mechanism is used to decode the hidden representation of the original given sentence and the aspect category decoder template sentence to obtain the output representation of the attention mechanism;

[0034] Use the feedforward layer to normalize the output of the attention mechanism;

[0035] The decoder template sentence corresponding to the aspect category with the highest score is used as the decoder output;

[0036] According to the aspect category decoder, the aspect category corresponding to the template sentence is taken as the final aspect category.

[0037] Furthermore, the predefined aspect sentiment polarity is used to generate the corresponding sentiment polarity decoder template sentence, and the encoded original given sentence is decoded to obtain the aspect sentiment corresponding to the aspect word, specifically:

[0038] Use the BART encoder to encode the original given sentence to obtain the hidden representation of the original given sentence;

[0039] Generate corresponding sentiment polarity decoder template sentences using predefined sentiment polarity;

[0040] The decoder's attention mechanism is used to decode the hidden representation of the original given sentence and the sentiment polarity decoder template sentence to obtain the output representation of the attention mechanism;

[0041] Use the feedforward layer to normalize the output of the attention mechanism;

[0042] The sentiment polarity decoder template sentence with the highest score is used as the decoder output;

[0043] The sentiment polarity corresponding to the template sentence of the sentiment polarity decoder is used as the final aspect sentiment.

[0044] According to some embodiments, a second solution of the present invention provides an aspect-level sentiment quadruple extraction system based on deep learning, which adopts the following technical solutions:

[0045] A deep learning-based aspect-level sentiment quadruple extraction system, including:

[0046] A sentence collection module is configured to obtain an original given sentence;

[0047] The sentiment quadruple extraction module is configured to extract sentiment quadruple based on the original given sentence using a pre-trained aspect-level sentiment quadruple extraction model;

[0048] Among them, the emotion quadruple extraction is performed using a pre-trained aspect-level emotion quadruple extraction model, including:

[0049] The aspect words in the original given sentence are extracted by using inline tags;

[0050] Constructing a question template sentence based on the extracted aspect words, and splicing the question template sentence with the original given sentence, and extracting opinion words paired with the aspect words based on the spliced sentence;

[0051] The predefined aspect categories and predefined aspect sentiment polarities are used to generate corresponding aspect category decoder template sentences and sentiment polarity decoder template sentences, and the encoded original given sentences are decoded to obtain aspect categories and aspect sentiments.

[0052] According to some embodiments, a third aspect of the present invention provides a computer-readable storage medium.

[0053] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the aspect-level sentiment quadruple extraction method based on deep learning as described in the first aspect above.

[0054] According to some embodiments, a fourth aspect of the present invention provides a computer device.

[0055] A computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the method for extracting aspect-level sentiment quadruple based on deep learning as described in the first aspect above are implemented.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] The present invention proposes a two-stage framework to enhance the correlation between aspects and opinions and fully utilize the semantic information of sentiment elements. Specifically, in the first stage, we regard the extraction task as a machine translation (MRC) problem. Treating it as a machine translation problem can convert it into a sequence-to-sequence task. Using this unified sequence-to-sequence task can avoid the error propagation problem in the processing of each subtask; the span-based tagging scheme can better handle multi-word aspect words and opinion words compared to the previous BIO tagging scheme, and it also has better performance; finally, it is constructed into a question-answering-based machine reading comprehension task to achieve effective extraction of aspect-opinion pairs. In the second stage, the classification of aspect categories and sentiment polarity is regarded as a text generation task. By learning to generate sentiment elements in natural language form, the semantics of sentiment elements can be fully utilized. At the same time, in recent years, generative algorithms have achieved the best performance in the task of aspect-level sentiment analysis. Therefore, treating the classification task of aspect categories and sentiment polarity as a generative task can greatly improve the performance of the model. Finally, the two stages are combined, resulting in a model that combines the excellent performance of an end-to-end algorithm with the easily understandable interpretability of a two-stage task. Combined with our proposed template generator, the model can decode aspect-sentiment quadruple. Extensive experiments on real-world datasets validate the effectiveness of our method, demonstrating superior results to comparable methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0059] Figure 1 This is a flow chart of a method for extracting aspect-level sentiment quadruple based on deep learning in an embodiment of the present invention;

[0060] Figure 2 1 is a schematic structural diagram of an aspect-level sentiment quadruple extraction model according to an embodiment of the present invention;

[0061] Figure 3 It is a structural diagram of a generation module in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0063] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0064] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0065] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0066] Example 1

[0067] like Figure 1 As shown, this embodiment provides an aspect-level sentiment quadruple extraction method based on deep learning. This embodiment uses the method applied to a server as an example for illustration. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal, a server, and a server, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:

[0068] Step S01: Obtain the original given sentence;

[0069] Step S02: Based on the original given sentence, sentiment quadruple extraction is performed using a pre-trained aspect-level sentiment quadruple extraction model;

[0070] In step S02, emotion quadruple extraction is performed using a pre-trained aspect-level emotion quadruple extraction model, including:

[0071] Step S021: extracting aspect words from the original given sentence using inline tags;

[0072] Step S022: constructing a question template sentence based on the extracted aspect words, and concatenating the question template sentence with the original given sentence, and extracting opinion words paired with the aspect words based on the concatenated sentence;

[0073] Step S023: Generate corresponding aspect category decoder template sentences and emotion polarity decoder template sentences using predefined aspect categories and predefined aspect emotion polarities, and decode according to the encoded original given sentence to obtain aspect categories and aspect emotions.

[0074] The sentiment quadruple is composed according to the extracted aspect words, opinion words, aspect categories and aspect sentiments.

[0075] Specifically, the specific implementation steps of the present invention are as follows:

[0076] like Figure 2 As shown, the overall model block diagram is introduced - aspect-level sentiment quadruple extraction model

[0077] The model of this embodiment consists of two extractors and two classifiers: the extractor on the left is designed to extract all aspect words in a given sentence, the extractor on the right is designed to extract all opinion words paired with aspect words in the sentence and combine them into aspect-opinion pairs, and the classifier is designed to classify the aspect category and sentiment polarity of the aspect-opinion pairs obtained by the extractor. Figure 3 As shown in Figure 2, the model also includes a generation module for generating task-specific questions or templates. The details of each component of the model will be given in the following content.

[0078] Problem Definition

[0079] Given a sentence, S={x1,x2,…,x n}, the goal of this embodiment is to obtain all the sentiment quadruples contained in the sentence, namely {C1, A1, O1, P1},…, {C n ,A n ,O n ,P n}, where C represents aspect category, A represents aspect word, O represents opinion word, and P represents aspect sentiment.

[0080] It's important to note that a sentence often contains multiple aspect and opinion words. The quadruple extraction task requires not only identifying the four elements but also combining them into a valid quadruple, while also considering the implicit aspects / opinions. Since implicit aspects / opinions cannot be explicitly expressed as a single word or phrase, in the case of implicit aspects, A is set to empty, and category C is used to describe opinion words. In the case of implicit opinions, O is set to empty, and sentiment S is used to describe their semantic orientation.

[0081] In step S021, the aspect words in the original given sentence are extracted by using inline tags, specifically:

[0082] Based on the original given sentence, the BERT encoder is used to obtain the embedded semantic representation of the original given sentence;

[0083] Based on the embedded semantic representation, the flag with a predicted probability greater than a set value is regarded as the beginning or end of the aspect element;

[0084] Pair each beginning token with its nearest ending token to determine an aspect word;

[0085] By analogy, it is determined to extract all aspect words in the original given sentence.

[0086] Specifically, the encoding module

[0087] BERT is the most commonly used encoder for extracting context-dependent features for downstream tasks. In the model of this embodiment, BERT is used as both a sentence encoder and a joint encoder. In the sentence encoder, BERT is used to encode the context-dependent features of a given sentence; in the joint encoder, BERT is used to encode the auxiliary question and original sentence pairs constructed by the question generation module. Through BERT's bidirectional self-attention mechanism, prior information about aspect words can be encoded into the representation of the original sentence, allowing for the subsequent extraction of relevant opinion words.

[0088] Before aspect word extraction, BERT encoder is used to obtain the embedded semantic representation of the original review text S. A represents the aspect extraction task, and then the Span method is used to extract all aspect words in the comment text.

[0089] Aspect word extraction module - extracts aspect words from the encoded embedded semantic representation

[0090] Specifically, in the training phase, this embodiment uses the Span method to extract all aspect words in the review text, that is, two binary classifiers are used to predict the start and end positions of the aspect words. The specific implementation is as follows:

[0091]

[0092]

[0093] in, and Respectively represents the probability that the i-th or j-th word is the beginning or end of an aspect word, A represents an aspect word, s and e represent the beginning and end of an aspect word, respectively, and w As and w Ae is a learnable matrix, b As and b Ae It is a deviation. It should be noted that, because the starting position of the aspect word must be before the ending position, it is necessary to ensure that i <j。

[0094] Next, we define all tokens with a probability greater than 0.5 as the start or end of an aspect element. We then pair each start token with its nearest end token and treat them as an aspect word to determine all aspect words in the sentence.

[0095] The loss function for the aspect word extraction task can be implemented using binary cross entropy loss between the predicted value and the true label, as follows:

[0096]

[0097] in, represents the total loss of aspect word extraction, and Respectively represent the losses of the start and end positions of the aspect words, BCE represents the binary cross entropy loss function, and Indicate the start and end of the true label respectively.

[0098] In the specific practical application process, after using the BERT encoder to obtain the embedded semantic representation of the original comment text S, based on all tokens with a probability greater than 0.5 as the beginning or end of the aspect element, each token as the beginning in the embedded semantic representation is paired with its nearest token as the end, and treated as an aspect word to determine all aspect words in the sentence.

[0099] In step S022, a question template sentence is constructed based on the extracted aspect words, and the question template sentence is concatenated with the original given sentence, including:

[0100] Construct question template sentences using fixed template sentences based on aspect words;

[0101] Based on the question template statement, the original given statement is spliced;

[0102] Get the concatenated statement.

[0103] The extraction of opinion words paired with aspect words based on the concatenated sentences is specifically as follows:

[0104] Based on the concatenated sentences, the BERT encoder is used to obtain the opinion representation of the original given sentence;

[0105] Opinion representation based on the original given sentence, using the flag whose predicted probability is greater than the set value as the beginning or end of the opinion word;

[0106] Pair each start token with its nearest end token to determine an opinion word;

[0107] By analogy, it is determined to extract all opinion words in the opinion representation of the original given sentence.

[0108] Question Generator - Embed the prior information of the aspect extraction task into the opinion word extraction task

[0109] Specifically, the question generator solves the problem of transferring the results of the aspect extraction task to the opinion word extraction task, connecting the two subtasks, so as to better extract opinion words and at the same time achieve the paired matching of aspect opinions. The opinion word extraction task is transformed into a MRC problem instead of a sequence labeling problem. Therefore, the goal of the question generator is to generate appropriate question sentences to better adapt to the opinion extraction task. For simplicity, a fixed template sentence "How about the [aspect]?" is used. Through such a template, such a template sentence can be created for each aspect word in the comment sentence.

[0110] Opinion Word Extraction Module - Extract opinion words from the template sentences containing aspect information

[0111] As Figure 2 shown, the template sentence generated by the previous question generator is connected to the original comment text and then fed into the Join BERT encoder. Through the bidirectional cross-attention in BERT, the prior information of the aspect extraction task can be fully encoded into the representation of the original sentence, and the opinion representation of the comment sentence can be obtained through BERT That is to say, by jointly encoding the auxiliary question and the original sentence pair, the prior information of the aspect word is encoded into the representation of the original sentence in order to obtain the opinion representation for a specific aspect word.

[0112] Similar to the previous aspect word extraction, the opinion word extraction also uses a span-based method. In the training stage, two binary classifiers are used to predict the start and end positions of the opinion word. The specific implementation is as follows:

[0113]

[0114]

[0115] Among them, and respectively represent the probabilities that the i-th or j-th word is the start or end of an opinion word. w Os and w Oe are learnable matrices, and b Os and b Oe are biases. Similarly, i < j, and the tokens with probabilities greater than 0.5 are regarded as the start or end of an opinion word. The loss function of the opinion extraction task also uses binary cross-entropy loss, specifically:

[0116]

[0117] in, represents the total loss of opinion word extraction, and Respectively represent the loss of the starting position and the ending position of the opinion word, BCE represents the binary cross entropy loss function, and Indicate the start and end of the true label respectively.

[0118] In specific practical applications, after using the Join BERT encoder to obtain the opinion representation of the comment sentence, based on the assumption that all tokens with a probability greater than 0.5 are the beginning or end of an opinion word, each starting token in the opinion representation of the comment sentence is paired with its nearest ending token as an opinion word to determine all opinion words in the sentence.

[0119] After the above tasks, all aspect-opinion pairs are obtained.

[0120] In step S023, the corresponding aspect category decoder template sentence is generated using the predefined aspect category, and decoded according to the encoded original given sentence to obtain the aspect category corresponding to the aspect word, specifically:

[0121] Use the BART encoder to encode the original given sentence to obtain the hidden representation of the original given sentence;

[0122] Generate corresponding aspect category decoder template sentences using predefined aspect categories;

[0123] The decoder's attention mechanism is used to decode the hidden representation of the original given sentence and the aspect category decoder template sentence to obtain the output representation of the attention mechanism;

[0124] Use the feedforward layer to normalize the output of the attention mechanism;

[0125] The decoder template sentence corresponding to the aspect category with the highest score is used as the decoder output;

[0126] According to the aspect category decoder, the aspect category corresponding to the template sentence is taken as the final aspect category.

[0127] In step S023, the predefined aspect sentiment polarity is used to generate a corresponding sentiment polarity decoder template sentence, and the encoded original given sentence is decoded to obtain the aspect sentiment corresponding to the aspect word, specifically:

[0128] Use the BART encoder to encode the original given sentence to obtain the hidden representation of the original given sentence;

[0129] Generate corresponding sentiment polarity decoder template sentences using predefined sentiment polarity;

[0130] The decoder's attention mechanism is used to decode the hidden representation of the original given sentence and the sentiment polarity decoder template sentence to obtain the output representation of the attention mechanism;

[0131] Use the feedforward layer to normalize the output of the attention mechanism;

[0132] The sentiment polarity decoder template sentence with the highest score is used as the decoder output;

[0133] The sentiment polarity corresponding to the template sentence of the sentiment polarity decoder is used as the final aspect sentiment.

[0134] Template Generator - Converts classification tasks into generation tasks

[0135] The template generator integrates the resulting aspect-opinion pairs into the subsequent classification task, enabling the classification task to better capture more useful information and achieve more accurate classification. Therefore, the two classification tasks are transformed into generation tasks. Therefore, the goal of the template generator is to generate a suitable template to better suit the generation task. The aspect category template generated by the template generator contains a slot for a given aspect category label, and the aspect sentiment template generated contains a slot for a given aspect category label and another slot for a sentiment polarity type label.

[0136] Define the set C = {c1, c2, ..., cm} to represent the predefined aspect categories, where m represents the number of predefined aspect categories. In addition, define the set P = {p1, p2, ..., pn} to represent the aspect sentiment polarity, where n is equal to 3, representing "positive", "negative", and "neutral" respectively.

[0137] like Figure 3 As shown, these two sets are used to define the aspect category decoder template and the aspect emotion polarity decoder template respectively. For simplicity, both templates use fixed template statements, where It is defined as “The [A] is belong to [C]”, where [A] represents the corresponding aspect word. is defined as “The sentimentpolarity of [C] is [P]”, where The construction needs Decoding.

[0138] like Figure 2 、 Figure 3 As shown, the aspect category decoder

[0139] For a given aspect word, first create all possible aspect category templates The original review is then fed into the BART encoder to obtain the hidden representation of the sentence, namely:

[0140] h enc =ENCODER(S)

[0141] Then, in the decoder step c, h enc and the output of step c-1 as the input of the decoder to obtain an output representation using the attention mechanism, that is,

[0142]

[0143] After that, the obtained representation is fed into the feed-forward layer and normalized to obtain its accuracy score:

[0144]

[0145] in, Indicates the vocab size of pre-trained BART.

[0146] Finally, the template with the highest score is selected as the output of the decoder, and the aspect category corresponding to the aspect word is finally decoded. The cross entropy loss is used as the loss function between the output of the decoder and the original template:

[0147]

[0148] in, represents the loss function for aspect category classification, t c ∣t 1,c-1 represents the output of the BART decoder at step c, and C represents the true aspect category.

[0149] like Figure 2 As shown, the emotion polarity decoder

[0150] After getting the aspect categories through the aspect category decoder, create all possible sentiment polarity templates The trained model is then used to score each template, similar to the aspect category decoder. The template with the highest score is then selected as the decoder output, and the sentiment polarity is finally decoded. For sentiment polarity classification, we also use cross entropy loss as the loss function:

[0151]

[0152] in, represents the loss function for sentiment polarity classification, t c ∣t1，c-1 represents the output of the BART decoder at step c, and S represents the true sentiment polarity.

[0153] joint training

[0154] Finally, the overall loss function can be expressed as:

[0155]

[0156] Among them, λ is a hyperparameter used to determine the contribution of each task.

[0157] This embodiment discloses a two-stage aspect-level sentiment quadruple extraction method enhanced by a pre-trained language model. A two-stage framework is used in ASQP, first extracting aspect-opinion pairs, and then classifying aspect categories and sentiment polarity. The combination of subtasks has good performance and is consistent with human cognition. We formulate the aspect-opinion pair extraction task as an MRC problem rather than a sequence labeling problem. By solving the extraction task from this perspective, the model of this embodiment can better capture the prior characteristics specified by the aspects and has excellent interpretability. At the same time, this design can alleviate the negative effects of error propagation and redundant pairs faced by token extraction-then-classification methods. By considering the classification task (i.e., aspect sentiment classification and aspect category classification) as a text generation task, the model of this embodiment can better learn the correlation between input and output templates compared to traditional classification methods. By combining the template generator proposed in this embodiment, the classification effect is better than the classification method using the same pre-trained model. A comprehensive analysis of the model and its different components was conducted, and the proposed model was evaluated on two commonly used datasets. The model of this embodiment outperformed the comparison method.

[0158] Comparative experiment

[0159] This example conducts experiments on two public datasets, Rest15 and Rest16. These two datasets originate from the SemEval task and were gradually annotated, aligned, and completed by previous researchers. Each instance in the two datasets contains a review sentence with one or more sentiment quadruplets.

[0160] The statistical data are shown in Tables 1 and 2 below. Table 1 shows the data statistics of the two datasets used in this embodiment, including their training set, validation set, and test set. Table 2 compares the performance of this embodiment with other mainstream models, using accuracy, recall, and F1 score as evaluation indicators.

[0161] In Table 1, #S, #+, #0, and #- represent the number of sentences, the number of positive, neutral, and negative quadruples, respectively.

[0162] Table 1 Experimental data statistics of different data sets

[0163]

[0164] Comparative test

[0165] Table 1 compares the evaluation results with the baseline methods in terms of precision (Pre, %), recall (Rec, %), and F1 score (F1, %).

[0166] Table 2 Performance comparison results of different models

[0167]

[0168]

[0169] From the experimental results in Table 2, it can be seen that this embodiment is superior to other comparison methods in terms of accuracy, recall rate and F1 score, which means that this embodiment can better handle the problem of aspect sentiment quadruple extraction.

[0170] Example 2

[0171] This embodiment provides an aspect-level sentiment quadruple extraction system based on deep learning, including:

[0172] A sentence collection module is configured to obtain an original given sentence;

[0173] The sentiment quadruple extraction module is configured to extract sentiment quadruple based on the original given sentence using a pre-trained aspect-level sentiment quadruple extraction model;

[0174] Among them, the emotion quadruple extraction is performed using a pre-trained aspect-level emotion quadruple extraction model, including:

[0175] The aspect words in the original given sentence are extracted by using inline tags;

[0176] Constructing a question template sentence based on the extracted aspect words, and splicing the question template sentence with the original given sentence, and extracting opinion words paired with the aspect words based on the spliced sentence;

[0177] The corresponding aspect category decoder template sentence and emotion polarity decoder template sentence are generated by using the predefined aspect category and predefined aspect emotion polarity, and are decoded according to the encoded original given sentence to obtain the aspect category and aspect emotion.

[0178] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment 1. It should be noted that the above modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0179] The descriptions of the various embodiments in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0180] The proposed system can be implemented in other ways. For example, the system embodiment described above is merely illustrative. For example, the above module division is only a logical function division. In actual implementation, other division methods may be used. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not implemented.

[0181] Example 3

[0182] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the method for extracting aspect-level sentiment quadruple based on deep learning as described in the first embodiment above are implemented.

[0183] Example 4

[0184] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the aspect-level sentiment quadruple extraction method based on deep learning as described in the first embodiment are implemented.

[0185] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0186] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0187] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0188] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0189] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0190] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A method for extracting aspect-level sentiment quadruple based on deep learning, characterized by: include: Get the original given statement; Based on the original given sentence, sentiment quadruple extraction is performed using the pre-trained aspect-level sentiment quadruple extraction model; Among them, the emotion quadruple extraction is performed using a pre-trained aspect-level emotion quadruple extraction model, including: The aspect words in the original given sentence are extracted using inline tags, specifically: Based on the original given sentence, the BERT encoder is used to obtain the embedded semantic representation of the original given sentence; Based on the embedded semantic representation, the flag with a predicted probability greater than a set value is regarded as the beginning or end of the aspect element; Pair each beginning token with its nearest ending token to determine an aspect word; And so on, determine to extract all the aspect words in the original given sentence; Constructing a question template sentence based on the extracted aspect words, and splicing the question template sentence with the original given sentence, and extracting opinion words paired with the aspect words based on the spliced sentence; The corresponding aspect category decoder template sentence and emotion polarity decoder template sentence are generated using the predefined aspect category and predefined aspect emotion polarity, and decoded according to the encoded original given sentence to obtain the aspect category and aspect emotion, specifically: Use the BART encoder to encode the original given sentence to obtain the hidden representation of the original given sentence; Using the predefined aspect categories and the predefined aspect sentiment polarities, generating corresponding aspect category decoder template sentences and sentiment polarity decoder template sentences; Using the decoder's attention mechanism, the hidden representation of the original given sentence and the aspect category decoder template sentence, as well as the hidden representation of the original given sentence and the sentiment polarity decoder template sentence are decoded to obtain the corresponding output representation of the attention mechanism; The output representations of the two attention mechanisms are normalized and scored using the feedforward layer; The aspect category decoder template sentence and the sentiment polarity decoder template sentence corresponding to the highest score are used as the output of the corresponding decoder; According to the aspect category decoder template sentence corresponding to the aspect category and the sentiment polarity decoder template sentence corresponding to the sentiment polarity, the final aspect category and the final aspect sentiment are used.

2. The method for extracting aspect-level sentiment quadruple based on deep learning according to claim 1, characterized in that: The sentiment quadruple is composed according to the extracted aspect words, opinion words, aspect categories and aspect sentiments.

3. The method for extracting aspect-level sentiment quadruple based on deep learning according to claim 1, wherein: Construct a question template sentence based on the extracted aspect words, and use the question template sentence and the original given sentence for splicing, including: Construct question template sentences using fixed template sentences based on aspect words; Based on the question template statement, the original given statement is spliced; Get the concatenated statement.

4. The method for extracting aspect-level sentiment quadruple based on deep learning according to claim 1, wherein: The extraction of opinion words paired with aspect words based on the concatenated sentences is specifically as follows: Based on the concatenated sentences, the BERT encoder is used to obtain the opinion representation of the original given sentence; Opinion representation based on the original given sentence, using the flag whose predicted probability is greater than the set value as the beginning or end of the opinion word; Pair each start token with its nearest end token to determine an opinion word; By analogy, it is determined to extract all opinion words in the opinion representation of the original given sentence.

5. A deep learning-based aspect-level sentiment quadruple extraction system, characterized by: include: A sentence collection module is configured to obtain an original given sentence; The sentiment quadruple extraction module is configured to extract sentiment quadruple based on the original given sentence using a pre-trained aspect-level sentiment quadruple extraction model; Among them, the emotion quadruple extraction is performed using a pre-trained aspect-level emotion quadruple extraction model, including: The aspect words in the original given sentence are extracted using inline tags, specifically: Based on the original given sentence, the BERT encoder is used to obtain the embedded semantic representation of the original given sentence; Based on the embedded semantic representation, the flag with a predicted probability greater than a set value is regarded as the beginning or end of the aspect element; Pair each beginning token with its nearest ending token to determine an aspect word; And so on, determine to extract all the aspect words in the original given sentence; Constructing a question template sentence based on the extracted aspect words, and splicing the question template sentence with the original given sentence, and extracting opinion words paired with the aspect words based on the spliced sentence; The corresponding aspect category decoder template sentence and emotion polarity decoder template sentence are generated using the predefined aspect category and predefined aspect emotion polarity, and decoded according to the encoded original given sentence to obtain the aspect category and aspect emotion, specifically: Use the BART encoder to encode the original given sentence to obtain the hidden representation of the original given sentence; Using the predefined aspect categories and the predefined aspect sentiment polarities, generating corresponding aspect category decoder template sentences and sentiment polarity decoder template sentences; Using the decoder's attention mechanism, the hidden representation of the original given sentence and the aspect category decoder template sentence, as well as the hidden representation of the original given sentence and the sentiment polarity decoder template sentence are decoded to obtain the corresponding output representation of the attention mechanism; The output representations of the two attention mechanisms are normalized and scored using the feedforward layer; The aspect category decoder template sentence and the sentiment polarity decoder template sentence corresponding to the highest score are used as the output of the corresponding decoder; According to the aspect category decoder template sentence corresponding to the aspect category and the sentiment polarity decoder template sentence corresponding to the sentiment polarity, the final aspect category and the final aspect sentiment are used.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method for extracting aspect-level sentiment quadruple based on deep learning as described in any one of claims 1 to 4 are implemented.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps in the aspect-level sentiment quadruple extraction method based on deep learning as described in any one of claims 1-4 are implemented.

Images