Implicit aspect level sentiment analysis prediction method and system based on double prompt templates

Abstract

The invention relates to an implicit aspect level sentiment analysis and prediction method and system based on double prompt templates. The method comprises the steps that the double prompt templates are set for collected texts to conduct statement processing respectively; after statement processing, feature coding is carried out through two T5 encoders, corresponding sentence representation is obtained, and tag template features are constructed for a double-prompt template; in a decoding stage, two T5 decoders are used for interactively learning the dependency of the template sequence of the opposite side by utilizing the constructed label template characteristics, and the sentence representation of the double-prompt template is decoded and output respectively; and taking an intersection of two decoding outputs of the double-prompt template to obtain a final prediction result. Compared with the prior art, the method has the advantages of enhancing the dependence among the emotion elements, enhancing the capturing capability of implicit emotions, realizing more accurate emotion analysis and prediction and the like.

Description

Technical Field

[0001] The present invention relates to an aspect-based sentiment analysis method, and more particularly to an implicit aspect-level sentiment analysis prediction method and system based on a dual prompt template. Background Art

[0002] With the rapid development and popularization of contemporary communication technologies such as 5G mobile communication, broadband communication, digital communication, and the Internet, the production and life of the public have undergone great changes. The public has published a large amount of content and opinions on social platforms, e-commerce platforms, and streaming media platforms by means of these technologies. The information carried by these text data contains people's different views on current events, products, and video programs. If these data can be analyzed effectively, quickly, and accurately, for example, in e-commerce reviews, buyers often view relevant reviews from the page of the purchased product to help them decide whether it is necessary to buy the product. For merchants, if they can effectively analyze the reviews of buyers to know the attitudes and views of buyers towards their own products, then the service of the merchants will be greatly improved and the quality of the products will be enhanced; on social platforms, such as Weibo reviews, if the comments of interactors can be accurately analyzed, then it can help government departments understand the attitudes of the public towards current events, so as to correctly guide public opinion and thus facilitate the work of the government; on streaming media platforms, if the bullet screen comments of users can be captured and analyzed, then some incorrect remarks can be effectively avoided, and at the same time, the degree of preference of users for the current video can be understood. The previous analysis methods were nothing more than making judgments in an artificial form, which not only affected the efficiency but also could not achieve particularly good results.

[0003] Text sentiment analysis is a natural language processing (NLP) technology mainly used to analyze the sentiment polarity of a given text data, and is currently widely applied to services such as customer purchase reviews, questionnaire survey responses, online social media, and medical insurance. Sentence-level tasks mainly conduct sentiment polarity analysis at the sentence level, with a relatively small scope. However, for example, in an e-commerce review like "The product quality in this store is great, but the service attitude is very poor", it is difficult to know whether the sentiment expressed by this sentence is positive or negative if analyzed at the sentence level, which seems rather rough. But if analyzed at the aspect level, this sentence expresses the opinion tendencies of the two aspect words "product" and "service". "Product" is positive while "service" is negative. The aspect-level sentiment analysis task is different from general sentiment analysis tasks. It often requires knowing different aspects of a sentence and conducting polarity analysis on different aspects. In the early development of aspect sentiment analysis, due to the lack of data and technical limitations, the analysis results were not very accurate. Later, it could be further analyzed through sentiment dictionaries or personnel with sentiment knowledge based on specific rules. This method not only requires maintaining a sentiment knowledge dictionary but also consumes the time and energy of professionals. With the emergence of large datasets, machine learning techniques have been applied to sentiment analysis. Although it can save manpower to some extent, it still requires professionals to extract data features. Currently, automatically extracting features through neural networks and supplemented by deep learning techniques can significantly help researchers improve the performance of sentiment classification.

[0004] Existing research predicts sentiment elements through multi-template prompting methods to enhance the dependencies between elements, but it does not achieve information interaction in the generation process and also ignores the dependencies between the prompting templates and aspect terms and opinion terms in the input sequence. This paper proposes a dual-prompt-template mutual learning enhancement generation model to enhance information interaction between generation modules.

[0005] How to achieve accurate sentiment analysis prediction with information interaction and dependencies between sentiment elements has become a technical problem to be solved. Summary of the Invention

[0006] The purpose of the present invention is to provide an implicit aspect-level sentiment analysis prediction method based on dual-prompt templates to overcome the defects of the above-mentioned existing technologies.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] According to one aspect of the present invention, there is provided an implicit aspect-level sentiment analysis prediction method based on dual-prompt templates, the method comprising:

[0009] Set double - hint templates for the collected text and perform sentence processing respectively;

[0010] After sentence processing, perform feature encoding through two T5 encoders respectively to obtain corresponding sentence representations, and construct label template features for the double - hint templates;

[0011] In the decoding stage, utilize the constructed label template features, and the two T5 decoders interact to learn the dependencies of each other's template order, and decode and output the sentence representations of the double - hint templates respectively;

[0012] Take the intersection of the two decoding outputs of the double - hint templates to obtain the final prediction result.

[0013] Preferably, the double - hint templates include: and where is obtained by swapping the positions of in and [AT], [OT], [AC] and [SP] are all special markers used to represent the information structure of emotional elements.

[0014] Preferably, in the decoding stage, the objective is to minimize the cross - entropy as the generation loss, specifically: by calculating and accumulating the probabilities at each time step t, the generation loss

[0015]

[0016] p θ (H t+1 |H e ,H d＜t ) = softmax(W T H dt )

[0017] H dt = T5_Decoder(H d＜t ,H e )

[0018] where N represents the length of the sentence, θ is a hyperparameter, W is the transition matrix, θ is the initialized hyperparameter, T5_Decoder is the T5 decoder, p θ (H t+1 |H e ,H d<t ) is the conditional probability of calculating the decoding output H e at the t - th time step according to the context input H d<t and the output H dt before the t - th time step, softmax is the softmax function, H t+1is the hidden state of the decoder at the (t+1)-th time step.

[0019] Preferably, the process of constructing the label template features for the dual hint template includes:

[0020] Perform mean pooling on the encoded hidden layer H e to obtain the aspect term representation H AT and the representation H OT of the opinion term through a fully connected layer;

[0021] The aspect term label template feature F AT and the opinion term label template feature F OT are obtained through H AT and H OT in the encoding stage and HM AT and HM OT in the decoding stage, where HM AT and HM OT are the intermediate results of extracting the aspect term and opinion term features in the decoding stage, respectively, and contain the combined information of the decoder hidden layer features and the label position features;

[0022] Input the opinion term label template feature F AT and the aspect term label template feature F OT into a multi-layer perceptron to obtain two decoder aspect word and opinion word prediction label distributions for the dual hint template and respectively.

[0023] More preferably, the mutual learning loss of the two decoder interactive learning is defined as the KL divergence, specifically:

[0024]

[0025] where and are the aspect word and opinion word prediction label distributions output by the hint template respectively, and are the aspect word and opinion word prediction label distributions output by the hint template respectively, and are the mutual learning losses of the aspect word and opinion word respectively.

[0026] Preferably, in the decoding stage, it also includes adding auxiliary words and expanding the BIO labels, specifically:

[0027] In the training stage, auxiliary words are added to the input sequence to distinguish implicit aspect words and opinion words;

[0028] Expand the BIO tags, and set multiple types of tags to mark explicit or implicit different types of aspect words and opinion words in the sequence.

[0029] More preferably, in the decoding stage, use the last hidden layer H of the decoder dt as the representation of the tag, and calculate the tag features of the aspect term and the opinion term through the tag position P M :

[0030]

[0031] M AT = σ(W 1 HM AT + b 1 )

[0032] M OT = σ(W 1 HM OT + b 1 )

[0033] where W 1 is the weight matrix, b 1 is the bias, σ is the relu activation function, H dt is the last hidden layer of the decoder, represents the vector concatenation operation, and represent the tag position vectors of the aspect term and the opinion term respectively; HM AT and HM OT are the intermediate results of extracting the features of the aspect term and the opinion term respectively, containing the combined information of the decoder hidden layer features and the tag position features; M AT and M OT are the aspect tag feature set and the opinion tag feature set respectively;

[0034] The representation of the aspect word and the opinion word in the decoding stage: Pass the tag features M AT and M oT through a fully connected network to predict the BIO tags:

[0035]

[0036] where d represents the vector dimension, N represents the sequence length, W 2 is the weight matrix, b 2 is the bias, and are the prediction results of the aspect term and the opinion term respectively.

[0037] More preferably, the training loss function for predicting BIO tags is defined as cross-entropy, and the prompt template and the loss functions for the generation processes are respectively and

[0038]

[0039] where N is the sequence length, K is the number of quadruples in a data sequence, t is the token vector in the token set T, and respectively represent the predicted corresponding tokens that are aspect terms and opinion terms.

[0040] Preferably, in the decoding stage, the candidate list is dynamically adjusted according to the state of the current word vector;

[0041] If the current token is decoded as '[', the next token should be selected from a specific list of entries, and at the same time, the current entry is tracked to decode the next entry.

[0042] According to another aspect of the present invention, an implicit aspect-level sentiment analysis prediction system based on a dual prompt template is provided. The system includes a dual prompt template generation module, a mutual learning information enhancement module, and a label enhancement decoding module. The dual prompt template generation module includes a T5 encoder, a T5 decoder, and a defined dual prompt template;

[0043] The input text is processed by the dual prompt template, and feature encoding is performed through two T5 encoders to obtain corresponding sentence representations;

[0044] The mutual learning information enhancement module is used to construct label template features for aspect terms and opinion terms, and use the constructed label template features to enable the two decoders to interactively learn the dependencies of different template orders;

[0045] In the decoding stage, using the constructed label template features, the two T5 decoders interactively learn the dependencies of each other's template orders, and respectively decode and output the sentence representations of the dual prompt template; the intersection of the two decoding outputs of the dual prompt template is taken to obtain the final prediction result;

[0046] The label enhancement decoding module is used to share explicit sentiment expressions in different sequences during the decoding stage, including adding auxiliary words to the input sequence, expanding BIO tags according to different types and features of aspect words and opinion words, calculating label features for aspect terms and opinion terms, and predicting BIO tags.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1) The present invention designs a double prompt template based on prompt learning, constructs label template features for the double prompt template, interactively learns in the decoding stage to enhance the dependency between emotional elements, shares explicit emotional expressions in different sequences, can enhance the model's ability to capture implicit emotions, makes full use of label dependency to break the sequential decoding limitation of the decoder, iteratively interactively learns label information, and can more accurately analyze the emotional tendencies of specific aspects in the text.

[0049] 2) The present invention independently trains and optimizes the dual-prompt template generation unit, which can give full play to the advantages of the dual templates, take the intersection of the two outputs to obtain the final result, and improve the accuracy and reliability of the final prediction result.

[0050] 3) The present invention utilizes the constructed label template features to enable the two decoders to interactively learn the dependencies of different template orders, effectively share information and promote each other, and effectively improve the accuracy and stability of the prediction results.

[0051] 4) The present invention innovatively adds a label enhancement decoding module in the decoding stage, which can fully share the explicit emotional expressions in the sequence. By effectively utilizing the explicit emotional expressions, the model's ability to capture implicit emotions is greatly enhanced. When faced with complex text situations, it can more deeply explore potential emotional information, thereby significantly improving the performance and effect of aspect-based sentiment analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Schematic diagram of a prediction model based on a double prompt template in the present invention;

[0053] Figure 2 Schematic diagram of the interaction of the tag-based enhanced decoding module in the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0055] This embodiment relates to an implicit aspect-level sentiment analysis and prediction method based on a double-prompt template, which is dedicated to achieving accurate analysis and prediction of sentiment tendencies in specific aspects.

[0056] like Figure 1 ,Firstly, according to the disorder of the elements of the sentiment analysis ,quadruple, a double prompt template generation module is set up. Secondly, in the decoding stage, a label ,enhanced decoding module is set up to enhance the ability of capturing ,implicit words.

[0057] 1. Dual Prompt Template Generation Module

[0058] The dual prompt template generation module includes a dual prompt template, a T5 encoder, and a T5 decoder.

[0059] To effectively capture the emotional element dependencies in the sequence, this embodiment ingeniously introduces a prompt template based on the element order. On the one hand, the generation prompt template for a sentence is defined as where the special tokens [AT], [OT], [AC], and [SP] are used to represent the structure of the information. It should be noted that although the quadruple obtained from the text-based aspect-level sentiment analysis prediction is in an unordered state, the decoding process is sequential. On the other hand, to significantly enhance the model's perception ability of aspect terms and opinion terms, by swapping and while keeping the positions of other elements unchanged, another template is obtained. Moreover, to correctly identify these two templates, prefixes "ACSO:" and "OCSA:" are specifically added during input. If an input sequence contains multiple quadruples, the special symbol [SSEP] is used for connection.

[0060] This embodiment is based on the transformer architecture as the basic framework and deeply and fully utilizes the pre-trained model T5 to perform context encoding operations. The Transformer architecture has unique advantages. For example, its self-attention mechanism can effectively process long sequence data and shows excellent performance in many natural language processing tasks. By cleverly leveraging the language patterns and semantic information learned by the T5 pre-trained model during the pre-training process and applying them to the context encoding task, the input context information can be encoded more accurately, and the sentiment analysis prediction task is transformed into a generation task.

[0061] The implementation of the dual prompt template generation module includes the following steps:

[0062] 1-1. Define the prompt template

[0063] Carefully analyze the characteristics and requirements of the aspect-based sentiment analysis task, and determine that the generation prompt template for a sentence is where the special tokens [AT], [OT], [AC], and [SP] each play an important role in clearly representing the information structure. The design of these special tokens has undergone a large number of experiments and optimizations to ensure that the emotional elements in the text can be accurately captured. Swap and while keeping the positions of other elements unchanged to obtain the template At input, to correctly identify the two templates, prefixes "ACSO:" and "OCSA:" are added. If there are multiple quadruples in the input sequence, they are connected using the special symbol [SSEP].

[0064] 1-2. Encode using the pre-trained model

[0065] This embodiment is based on the transformer architecture and makes full use of the pre-trained model T5 for context encoding. Specifically, as Figure 1 , sentence W first passes through the T5 encoder (T5_Encoder) to obtain the sentence representation H e , that is, H e = T5_Encoder(W).

[0066] 1-3. Decoding process

[0067] In the decoding stage, the decoding output H dt at the t-th time step is jointly determined by multiple factors. The output H d<t before the t-th time step and the context input H e both affect the current decoding result. Through the precise calculation formula:

[0068] H dt = T5_Decoder(H d＜t , H e )

[0069] Among them, H dt can accurately calculate the decoding output at the current time step. In this process, the parameters of the T5 decoder need to be adjusted and optimized to ensure the accuracy and efficiency of decoding. Calculate the probability:

[0070] p θ (H t+1 |H e , H d＜t ) = softmax(W T H dt )

[0071] Among them, W is the transition matrix and θ is the initialized hyperparameter. This step requires reasonable initialization and adjustment of the transition matrix W and the hyperparameter θ to ensure the accuracy and reliability of probability calculation.

[0072] 1-4. Training generation loss

[0073] In the training stage, the goal is to minimize the cross-entropy as the generation loss. By calculating and accumulating the probabilities at each time step t, the generation loss Where N represents the length of the sentence and θ is a hyperparameter. During the training process, the model's parameters need to be continuously adjusted to minimize the generation loss and improve the model's performance. Prompt template and The generation processes respectively obtain the generation losses and By independently training and optimizing the two generation units, the advantages of the dual templates can be fully exploited. Finally, the intersection of the two outputs is taken to obtain the final result. This process requires a reasonable design and implementation of the calculation method for the intersection to ensure the accuracy and reliability of the final result.

[0074] 2. Mutual Learning Information Enhancement Module

[0075] The generation processes of different templates are independent of each other, and the model cannot share the decoded features. To achieve information interaction in the dual-template generation process, an enhanced training method based on label templates for mutual learning is designed, as Figure 1 shown. First, construct the label template features of aspect terms and opinion terms. Take the mean pooling of the encoded hidden layer and obtain the representations of aspect terms and opinion terms through a fully connected layer.

[0076] In this way, the two decoders can interactively learn the dependencies of different template orders. During the iterative process of training, the two decoders can conduct knowledge learning and promote each other. The mutual learning loss at this stage is defined as the KL divergence.

[0077] The implementation of the mutual learning information enhancement module includes:[[]]

[0078] 2-1. Construct label template features

[0079] First, perform mean pooling on the encoded hidden layer. This process requires an in-depth understanding of the structure and content of the hidden layer to ensure that the result of mean pooling can accurately reflect the encoded information. Then, obtain the representations of aspect terms and opinion terms through a fully connected layer. The parameters of the fully connected layer need to be reasonably initialized and adjusted to ensure that the obtained representations can accurately reflect the characteristics of aspect terms and opinion terms.

[0080] The construction of opinion term label template features and aspect term label template features needs to comprehensively consider the specific content in the encoding stage and the decoding stage. Through in-depth analysis and processing of this content, more accurate and useful label template features can be obtained.

[0081] Take the encoded hidden layer H e for mean pooling and obtain the representations of aspect terms and opinion terms through a fully connected layer:

[0082] H AT= MLP(Mean(H e ))

[0083] H OT = MLP(Mean(H e ))

[0084] where MLP is a fully connected layer and Mean is for mean pooling operation. Aspect term label template feature F AT and opinion term label template feature F OT are obtained through H AT 、H OT in the encoding stage, HM AT and HM OT in the decoding stage.

[0085]

[0086] where W 3 is the weight matrix, b 3 is the bias, and σ is the relu activation function. HM AT and HM OT are the intermediate results of extracting aspect term and opinion term features in the decoding stage, respectively, and contain the combined information of decoder hidden layer features and label position features.

[0087] Feed F AT and F OT into the multi-layer perceptron, and two decoder aspect word and opinion word prediction label distributions are obtained for the prompt templates and respectively.

[0088]

[0089] where and are the aspect word and opinion word prediction label distributions output by the prompt template respectively, and and are the aspect word and opinion word prediction label distributions output by the prompt template respectively.

[0090] 2 - 2, realizing decoder interactive learning

[0091] By using the constructed label template features, the two decoders can interactively learn the dependencies of different template orders. This process requires the design of a reasonable interactive learning mechanism to ensure that the two decoders can effectively share information and promote each other. During the iterative process of training, the two decoders learn knowledge and promote each other. The mutual learning loss is defined as the KL divergence. By calculating and optimizing the KL divergence, the stability and effectiveness of the two decoders in the interactive learning process can be ensured. The mutual learning loss is defined as follows:

[0092]

[0093] 3. Label enhanced decoding module

[0094] In the decoding stage, a label enhancement decoding module is innovatively added. This module can fully share the explicit emotional expressions in the sequence, and through the effective use of explicit emotional expressions, it greatly enhances the model's ability to capture implicit emotions. When faced with complex text situations, it can more deeply explore potential emotional information, thereby significantly improving the performance and effect of aspect-based sentiment analysis. For example, for some texts with more implicit emotions, this module can better reveal the emotional tendencies therein.

[0095] The implementation of the tag enhanced decoding module includes:

[0096] 3-1, Add auxiliary words and expand BIO tags

[0097] During the training phase, an auxiliary word “NULL” is added to the input sequence to help effectively distinguish implicit aspect words and opinion words. Figure 1 and Figure 2 As shown, for sequences containing implicit or explicit aspect terms, an explicit suffix "NULL" is added, W = {w 1 ,w 2 ,...,w n ,NULL}, where the sequence length is N. [AT] and [OT] are special tags that contain prompt information of aspect terms and opinion terms during the generation process. The designed tags can help the model identify different element types and guide the location of element features. This process needs to be accurately implemented in the input processing module to ensure that the addition of auxiliary words does not affect the subsequent encoding and decoding process.

[0098] The BIO tag is extended and nine types of tags T = {B-EA, I-EA, B-IA, I-IA, O, B-EO, I-EO, B-IO, I-IO} are set. The design of these tags needs to fully consider the different types and characteristics of aspect words and opinion words to ensure that the explicit or implicit aspect words and opinion words of the sequence can be accurately marked.

[0099] 3-2, Calculate label features and enhance extraction ability

[0100] Use the last hidden layer H of the decoder dt As the representation of the label (marker), through the label position P M Calculate the label features of aspect terms and opinion terms:

[0101]

[0102] M AT = σ(W 1 HM AT + b 1 )

[0103] M OT = σ(W 1 HM OT + b 1 )

[0104] Among them, W 1 is the weight matrix, b 1 is the bias, σ is the relu activation function, H dt is the last hidden layer of the decoder, represents the vector concatenation operation, and represent the label position vectors of aspect terms and opinion terms respectively. HM AT and HM OT are the intermediate results of extracting the features of aspect terms and opinion terms respectively, containing the combined information of decoder hidden layer features and label position features. The number of quadruples in a sentence is K, are the aspect and opinion label feature sets respectively.

[0105] Representations of aspect words and opinion words in the decoding stage: Use the label features M AT and M OT to predict BIO labels through a fully connected network, where

[0106]

[0107] Among them, d represents the vector dimension, N represents the sequence length, W 2 is the weight matrix, b 2 is the bias, and are the prediction results of aspect terms and opinion terms respectively.

[0108] The aspect terms and opinion terms in multiple quadruples are independent of each other, but the aspect words or opinion words can be shared. For a sequence containing multiple quadruples, during the decoding phase, it can share explicit and implicit term features, and enhance the model's extraction ability for aspect terms and opinion terms through the interaction learning of BIO tags and Marker representations. This process requires designing a reasonable interaction learning mechanism to ensure that the BIO tags and label features can work effectively together to improve the model's extraction ability.

[0109] The training loss function for this process is defined as cross-entropy. For the prompt templates and in the generation process, we respectively obtain and

[0110]

[0111] where N is the sequence length, K is the number of quadruples in a data sequence, t is the token vector in the token set T, and respectively represent the predicted corresponding tokens for aspect terms and opinion terms.

[0112] The calculation process of

[0113] 4. Pattern Constraint Generation Module

[0114] 4-1. Dynamically Adjusting the Candidate List

[0115] During decoding, this module dynamically adjusts the candidate list according to the state of the current word vector, rather than searching the entire vocabulary to find the next word vector to decode. This can effectively avoid the model generating invalid sequences that do not match the expectations. This process requires designing a reasonable dynamic adjustment algorithm to ensure that the adjustment of the candidate list can accurately reflect the state and requirements of the current token.

[0116] 4-2. Processing of Specific Tokens

[0117] If the current token is decoded as '[', then the next token should be selected from a specific list of entries, namely the specific tokens [A], [O], [S], and [C]. At the same time, track the decoding of the next entry for the current entry. This process requires precise implementation of the processing logic for specific tokens, which can ensure the accuracy and effectiveness of the decoding process and improve the efficiency and quality of sentiment analysis.

[0118] In summary, through the collaborative action of the dual-prompt template generation module, the mutual learning information enhancement module, the label marking interactive learning module, and the pattern constraint generation module, the aspect-based sentiment analysis method of the present invention can more accurately analyze the sentiment tendency of specific aspects in the text, providing a more effective sentiment analysis tool for fields such as social media and online review analysis, customer feedback management, market research and competitive analysis, public opinion monitoring and crisis management, and intelligent customer service and chatbots.

[0119] This embodiment also relates to an implicit aspect-level sentiment analysis prediction system based on a dual-prompt template, as Figure 1 shown in the figure. The system includes a dual-prompt template generation module, a mutual learning information enhancement module, and a label enhancement decoding module. The dual-prompt template generation module includes a defined dual-prompt template, a T5 encoder, and a T5 decoder;

[0120] The input text is processed by the dual-prompt template, and feature encoding is performed through two T5 encoders to obtain corresponding sentence representations;

[0121] The mutual learning information enhancement module is used to construct label template features of aspect terms and opinion terms, and use the constructed label template features to enable the two decoders to interactively learn the dependencies of different template orders;

[0122] In the decoding stage, using the constructed label template features, the two T5 decoders interactively learn the dependencies of each other's template orders, and respectively decode and output the sentence representations of the dual-prompt template; the intersection of the two decoding outputs of the dual-prompt template is taken to obtain the final prediction result;

[0123] As Figure 2 shown in the figure, the label enhancement decoding module is used to share explicit sentiment expressions in different sequences during the decoding stage, including expanding BIO tags according to different types and features of aspect words and opinion words, adding auxiliary words in the input sequence, calculating label features of aspect terms and opinion terms, and predicting BIO tags.

[0124] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. An implicit aspect-level sentiment analysis prediction method based on a double-prompt template, characterized in that: The method includes: Set up double prompt templates for the collected text to process sentences separately; After sentence processing, two T5 encoders are used to perform feature encoding to obtain the corresponding sentence representation, and label template features are constructed for the double prompt template; In the decoding stage, using the constructed label template features, the two T5 decoders interactively learn the dependency of each other's template order and decode and output the sentence representation of the double prompt template respectively; The final prediction result is obtained by taking the intersection of the two decoded outputs of the dual-prompt template.

2. According to claim 1, the implicit aspect-level sentiment analysis prediction method based on double prompt template is characterized in that: The dual prompt template includes: and in For exchange middle and [AT], [OT], [AC] and [SP] are special tags used to represent the information structure of emotional elements.

3. The implicit aspect-level sentiment analysis prediction method based on double prompt template according to claim 1 is characterized in that: The decoding stage aims to minimize the cross entropy as the generation loss. Specifically, the generation loss is obtained by calculating and accumulating the probability of each time step t. p θ (H t+1 |H e ,H d＜t )=softmax(W T H dt ) H dt =T5_Decoder(H d＜t ,H e ) Where N is the length of the sentence, θ is a hyperparameter, W is the transfer matrix, θ is the initialization hyperparameter, T5_Decoder is the T5 decoder, and p θ (H t+1 |H e ,H d＜t ) is input according to the context H e and the output H before time step t d<t Calculate the decoded output H at time step t dt The conditional probability of , softmax is the softmax function, H t+1 is the hidden state of the decoder at the t+1th time step.

4. The implicit aspect-level sentiment analysis prediction method based on double-prompt template according to claim 1 is characterized in that: The process of constructing a label template feature for a double prompt template includes: For the encoded hidden layer H e Perform mean pooling and obtain the aspect term representation H through a fully connected layer AT and the representation of opinion terms H OT ; Aspect term label template feature F AT and opinion term label template feature F OT Through the encoding stage H AT , H OT and HM in the decoding stage AT and H.M. OT Get, where HM AT and H.M. OT They are the intermediate results of extracting the features of aspect terms and opinion terms in the decoding stage, respectively, and contain the combined information of the decoder hidden layer features and the label position features; The opinion term label template feature F AT and aspect term label template feature F OT Input into the multi-layer perceptron, for the double prompt template and The predicted label distributions of two decoder aspect words and opinion words are obtained respectively.

5. The implicit aspect-level sentiment analysis prediction method based on double-prompt template according to claim 4 is characterized in that: The mutual learning loss of the interactive learning of the two decoders is defined as KL divergence, specifically: in, and Respectively prompt template The output aspect words and opinion words predict label distribution, and Respectively prompt template The output aspect words and opinion words predict label distribution, and are the mutual learning losses for aspect words and opinion words, respectively.

6. The implicit aspect-level sentiment analysis prediction method based on double-prompt template according to claim 1 is characterized in that: The decoding stage also includes adding auxiliary words and expanding BIO tags, specifically: During the training phase, auxiliary words are added to the input sequence to distinguish implicit aspect words and opinion words; Extend the BIO tag and set multiple types of tags to mark different types of explicit or implicit aspect words and opinion words in the sequence.

7. The implicit aspect-level sentiment analysis prediction method based on double-prompt template according to claim 6 is characterized in that: In the decoding stage, the last hidden layer H of the decoder is used dt As the representation of the label, through the label position P M Calculate the label features of aspect terms and opinion terms: M AT =σ(W1HM AT +b1) M OT =σ(W1HM OT +b1) Among them, W1 is the weight matrix, b1 is the bias, σ is the relu activation function, H dt is the last hidden layer of the decoder, represents the vector concatenation operation, and Represents the label position vectors of aspect terms and opinion terms respectively; HM AT and H.M. OT It is the intermediate result of extracting the features of aspect terms and opinion terms respectively, and contains the combined information of decoder hidden layer features and label position features; M AT and M OT They are the aspect label feature set and opinion label feature set respectively; Representation of aspect words and opinion words in the decoding stage: The label feature M AT and M OT Predict BIO tags through a fully connected network: in, d represents the vector dimension, N represents the sequence length, W2 is the weight matrix, b2 is the bias, and These are the prediction results for aspect terms and opinion terms, respectively.

8. The implicit aspect-level sentiment analysis prediction method based on double-prompt template according to claim 7 is characterized in that: The training loss function for predicting BIO labels is defined as cross entropy, suggesting that the template and The loss functions of the generation process are and Where N is the sequence length, K is the number of quadruplets in a data sequence, and t is the tag vector in the tag set T. and They represent the predicted corresponding tags of aspect terms and opinion terms respectively.

9. The implicit aspect-level sentiment analysis prediction method based on double-prompt template according to claim 1 is characterized in that: In the decoding stage, the candidate list is dynamically adjusted according to the state of the current word vector; If the current token is decoded as '[', the next token should be selected from the specified list of terms, and at the same time, the current term is tracked to decode the next term.

10. An implicit aspect-level sentiment analysis prediction system based on a dual-prompt template, the system being used to implement the method according to any one of claims 1 to 9, the system comprising a dual-prompt template generation module, a mutual learning information enhancement module and a label enhancement decoding module, wherein the dual-prompt template generation module comprises a T5 encoder, a T5 decoder and a defined dual-prompt template; The input text is processed by the double prompt template for sentence processing, and feature encoded by two T5 encoders to obtain the corresponding sentence representation; The mutual learning information enhancement module is used to construct label template features of aspect terms and opinion terms, and use the constructed label template features to enable the two decoders to interactively learn the dependencies of different template orders; In the decoding stage, using the constructed label template features, the two T5 decoders interactively learn the dependency of each other's template order and decode and output the sentence representation of the double prompt template respectively; The two decoded outputs of the dual-prompt template are intersected to obtain the final prediction result; The label enhancement decoding module is used to share explicit sentiment expressions in different sequences in the decoding stage, including adding auxiliary words to the input sequence, extending the BIO labels according to different types and features of aspect words and opinion words, calculating label features of aspect terms and opinion terms, and predicting BIO labels.

Images