Full-view prompt generation and wrong case discrimination cross training model for aspect-level emotion triple extraction and model training method

Abstract

The invention discloses a full-view prompt generation and wrong case discrimination cross training model for aspect-level emotion triple extraction and a model training method. The invention discloses a full view prompt generation and wrong case discrimination cross training model for aspect-level emotion triple extraction. The full view prompt generation and wrong case discrimination cross training model comprises a generation model input embedding module, an emotion triple generation model, a discrimination model input embedding module, an emotion triple discrimination model and an emotion triple output module. According to the method, the generation model is guided to learn semantic information in the label emotion triples from different perspectives through label sequence prompt of the full view, generation error example characteristics of the generation model are learned through the discrimination model, and extraction results of the full view are discriminated and filtered so as to eliminate the emotion triples with the adjacency dependence mismatch problem, so that the efficiency of the whole view is improved. Meanwhile, generation errors caused by multi-view prediction convergence are avoided; through the alternative training collaborative optimization scheme of the generative model and the discriminant model, the precision of the model for integrally extracting the emotion triad is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer natural language processing, and in particular, to a model and a model training method for aspect-level sentiment triple extraction based on cross-training of a full-view prompt generation model and a wrong example discrimination model. Background Art

[0002] Sentiment analysis, also known as opinion mining, is an important branch in the field of natural language processing (NLP), aiming to automatically identify and extract the sentiment tendency contained in natural language texts through technical means. Among them, fine-grained aspect-level sentiment analysis (ABSA), especially the aspect-level sentiment triple extraction (ASTE) task, can accurately identify the "evaluation object-opinion expression-sentiment polarity" triple by establishing the mapping relationship between text semantic units and sentiment elements, and shows important application value in scenarios such as business decision support, product experience optimization, and public opinion supervision.

[0003] In the prior art, methods based on the generative architecture have been widely used to handle the aspect-level sentiment triple extraction task. However, the existing baseline methods based on the generative model still have problems such as aspect-opinion mismatch caused by adjacency dependence and the problem of multi-view prediction convergence in actual tests.

[0004] Therefore, this patent proposes a model and a training method based on cross-training of a full-view prompt generation model and a wrong example discrimination model to improve the accuracy of aspect-level sentiment triple extraction and provide a fine-grained solution for public opinion analysis. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the present invention proposes a model and a model training method for aspect-level sentiment triple extraction based on cross-training of a full-view prompt generation model and a wrong example discrimination model, which can solve the problems of incorrect generation of aspect terms and opinion words such as adjacency dependence in the aspect-level sentiment triple extraction task of the existing model, and the problem of multi-view prediction convergence.

[0006] The present invention is realized through the following technical solutions: A full-view prompt generation and wrong example discrimination cross-training model for aspect-level sentiment triple extraction, characterized by including:

[0007] A generation model input embedding module for converting the original data into a format suitable for the generation model;

[0008] A sentiment triple generation model, which uses a fine-tuned open-source encoder-decoder structure generation model as the sentiment triple generation model. The generation model takes the word embedding sequence obtained after being processed by the input embedding module as the input and outputs a natural languageized sentiment triple sequence that conforms to the sequential prompts in the input;

[0009] The discriminative model input embedding module converts the generation result of the generation model into the input of the discrimination task, and obtains the input word embedding sequence of the discriminative model;

[0010] The sentiment triple discriminative model uses a fine-tuned open-source encoder-decoder structure model as the discriminative model for sentiment triples. A classification head is added to the top of the model to adapt to the discrimination task. The discriminative model takes the word embedding sequence obtained after being processed by the input embedding module as the input and the classification of the discrimination result as the output;

[0011] The sentiment triple output module combines the outputs of the sentiment triple generation model and the sentiment triple discriminative model, filters out the sentiment triples judged to be incorrect, and forms the final prediction result.

[0012] Furthermore, in the generation model input embedding module, first add the text "ASTE:" as a prompt at the head of the original input data. Then, construct a prompt with a specific label order, concatenate the sequential prompt at the end of the text to form the input text St of the generation model, and perform word vectorization processing on the input text St to form the generation model word embedding sequence X composed of word tokens GEN , and then pass the word embedding sequence X GEN into the sentiment triple generation model as the input.

[0013] Furthermore, in the discriminative model input embedding module, the module needs to perform formatting, shuffling and reconstruction, and encoding operations on the output sequence of the generation model in sequence; during formatting, convert the sentiment triple text sequence generated by the generation model into a sentiment triple in a unified format; during shuffling and reconstruction, first aggregate all the sentiment triples in a unified format generated by the generation module for the same original input text St and the formatted labeled sentiment triples, then shuffle the order of all the aggregated sentiment triples, and then concatenate each sentiment triple after the original text, separated by the separator marker "[SEP]" to form the reconstructed input statement, and set a new label for the reconstructed input statement according to the label content; during encoding, perform word embedding on the reconstructed input, denoted as Sr, add the "[CLS]" marker at the head of Sr, add position embedding and segment embedding, and finally obtain the input word embedding sequence X of the discriminative model DIS , the discriminative model input embedding module passes the word embedding sequence X DIS into the sentiment triple discriminative model as the input.

[0014] Furthermore, the sentiment triple discriminative model takes the word embedding sequence of the reconstructed input text as the input, captures the global semantic features in the reconstructed input text through the multi-head self-attention mechanism, and discriminates whether the current given sentiment triple is a correct extraction of the original text.

[0015] Furthermore, in the sentiment triplet output module, all sentiment triplets generated by the generation module for the same original text are first summarized, and then these triplets are screened and evaluated through the discriminant model; for sentiment triples that are judged to be wrong and sentiment triples with missing components, this module eliminates them, and finally, the sentiment triples that are judged to be correct by the discriminant model are retained as the final output sentiment triplet set.

[0016] The present invention also provides a training method for the above-mentioned model, and the technical solution adopted by the method is: comprising the following steps:

[0017] (1) Dataset preprocessing: Before training the model, the index labels of the dataset are first converted into corresponding natural language words, and then the sentiment polarity is transferred to natural language;

[0018] (2) Training the sentiment triple generation model: For each entry in the dataset, construct a full-view prompt of the sentiment element label “[A]”, “[O]”, and “[S]”; sequentially connect the original text with the prompts of the six views of the sentiment element label, and add the task description prompt “ASTE:” to the text header to form six different inputs. Combine the six labels corresponding to the six views to train the sentiment triple generation model. The trained sentiment triple generation model generates a set of sentiment triplets in the corresponding order according to the prompts of each view;

[0019] (3) Using the generated results to train the discriminant model: For an entry in the data set, the full-view emotion triple set generated by taking it as input is collected and passed into the discriminant model input embedding module, which formats, shuffles, reconstructs, and encodes the emotion triples in the set to form the training sample format of the discriminant model. The emotion triple discriminant model is trained using these reconstructed samples as the training data set;

[0020] (4) Alternating training: After the emotion triplet generation model has been trained for multiple training cycles, the intermediate results generated by the current generation model are collected to train the emotion triplet discrimination model; after the emotion triplet discrimination model has been trained for multiple training cycles, the emotion triplet generation model is reversely guided by the feedback mechanism based on its discrimination results to pay attention to the training samples with generation errors, thereby generating higher quality triples; the two are trained alternately until the set target training rounds are reached.

[0021] Furthermore, during alternating training, the training set is divided into mutually non-overlapping subsets A and B, where subset A contains data with a proportion of α in the original training set, and subset B contains the remaining 1-α data; the alternating training phase includes the following three phases:

[0022] Phase I: At the initial stage of alternating training, use subset A as the initial training data for the sentiment triple generation model; after training for θ1 iteration rounds, pause the training of the sentiment triple generation model, use the text in subset B as the input of the sentiment triple generation model, collect the sentiment triples of the generation model for the text in subset B, and complete the matching of the generated sentiment triples with the true labels of subset B by the input embedding module of the discriminant model, and convert the generation results of the sentiment triple generation model into the training samples of the sentiment triple discriminant model; the sentiment triple discriminant model is trained on the training dataset transformed from subset B, and after θ2 iteration rounds, pause the training and enter Phase II;

[0023] Phase II: On the basis of the training in Phase I, use subset B as the training dataset of the sentiment triple generation model and continue to start training; after training for θ1 iteration rounds, pause the training of the sentiment triple generation model, collect the sentiment triple generation results of the generation model for subset A, and synthesize the training samples of subset A for the discriminant model to supplement the discriminant model training set; the sentiment triple discriminant model continues to resume training on the basis of the training in Phase I using the updated training dataset, and after θ2 iteration rounds, pause the training and enter Phase III;

[0024] Phase III: Combine the data of subset A and subset B, that is, use all the data of the original training set to continue training the sentiment triple generation model on the basis of the training in Phase II. After training for θ1 iteration rounds, stop the training, collect the sentiment triple generation results of the generation model for subset A and subset B, and synthesize the discriminant model training samples to supplement and update the discriminant model training set; the sentiment triple discriminant model continues to resume training on the basis of the training in Phase II using the updated training dataset, and after θ2 iteration rounds, stop the training to obtain the final model.

[0025] Furthermore, the feedback mechanism adopted by the discriminant model is: during the training process, the discriminant model calculates the confidence score of the sample according to the following formula, collects the samples with confidence scores lower than the threshold β to form a low-confidence sample set, and the samples involved in the low-confidence sample set are oversampled during the training process of the generation model in the next alternating training round;

[0026]

[0027] Among them, Conf(t|St) represents the confidence score of the discriminant model for the sentiment triple t of the sample St, z represents the final logical value of the discriminant model, h [CLS] represents the hidden state of the [CLS] label of the discriminant model, W represents the weight matrix, b represents the bias vector, and d B represents the word embedding dimension of the discriminant model.

[0028] The beneficial effects of the present invention are as follows: Based on the idea of simplicity after complexity, the present invention guides the generation model to learn the semantic information in the label sentiment triples from different perspectives through the label order prompt of the full view. The discriminant model learns the generation error example features of the generation model to discriminate and filter the extraction results of the full view, so as to eliminate the sentiment triples with adjacent dependence mismatch problems and avoid generation errors caused by the convergence of multi-view predictions. By innovatively designing an alternating training and collaborative optimization scheme for the generation model and the discriminant model, the overall accuracy of the model for extracting sentiment triples is effectively improved, providing a fine-grained solution for public opinion analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is the overall framework diagram of the model in the present invention;

[0030] Figure 2 is the schematic diagram of full view prompt generation in the present invention;

[0031] Figure 3 is the step diagram of the alternating training method in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] The technical solutions of the present invention will be described in detail below through non-limiting embodiments in combination with the drawings.

[0033] As shown in the attached Figure 1 figures, a full view prompt generation and error example discriminant cross-training model for aspect-level sentiment triple extraction, the overall architecture of the model includes the following five modules:

[0034] (1) Generation model input embedding module

[0035] Prepare the input for the sentiment triple generation model. It is used to convert the original data into a format suitable for the generation model.

[0036] Specifically, the input embedding module first adds the text "ASTE:" at the head of the original input data as a prompt, indicating that the model needs to perform the aspect-level sentiment triple extraction task. Then, a prompt with a specific label order is constructed. The text label "[A]" is used to represent the aspect term element in the sentiment triple, "[O]" is used to represent the opinion word element, and "[S]" is used to represent the sentiment polarity element. The arrangement order of the three labels represents the order of the expected generation target. For example, "[O][S][A]" means that when the model generates sentiment triples, the generation order is opinion word, sentiment polarity, and aspect term. Introducing the order prompt can guide the model to process the same input text from different perspectives and fully mine the semantic features of the existing data when the training data is limited. The input embedding module concatenates the order prompt at the end of the text to form the input text St=(w1,w2,...,w n ), where wi Represents the i-th word in the text. The input text St is vectorized into a sequence of word embeddings X composed of tokens for the generative model GEN :

[0037]

[0038] x i = embedding T5 (w i )

[0039] where w i ' represents the i-th token after word segmentation, and embedding T5 (·) represents the word embedding model of the pre-trained generative model, and x i represents the word embedding of the i-th token. m represents the total number of tokens after input word segmentation, and d represents the embedding dimension of a single token.

[0040] The input embedding module of the generative model passes the word embedding sequence X GEN to the sentiment triple generation model as input.

[0041] (2) Sentiment triple generation model

[0042] Adopts a fine-tuned open-source encoder-decoder structure generation model (such as the T5 model) as the sentiment triple generation model. The generation model takes the word embedding sequence obtained after being processed by the input embedding module as input and outputs a natural language sentiment triple sequence. The training objective of the model is to maximize the similarity between the aspect-level sentiment triple sequence generated by the model and the true triple sequence, and uses the cross-entropy loss function to optimize the model.

[0043] The model output target is a natural language sentiment triple label sequence that conforms to the sequential prompt in the input. Let the set of sentiment triple labels corresponding to the text St be

[0044] where represents the i-th labeled sentiment triple, and T g represents the set of labeled sentiment triples; taking the input with the sequential prompt "[O][S][A]" as an example, the corresponding output target is the text sequence Go:

[0045]

[0046] This sequence uses [SSEP] as the label separator to segment sentiment triples that conform to a specific order, and adds corresponding labels to the elements within each sentiment triple for decoding.

[0047] Let the model output embedding be where, yi is the word embedding representation of a single token in the output, and m' represents the total number of tokens in the model output; the embedding of the label Go is where g i represents the word embedding of a single token in the label, and m g represents the total number of label tokens; the loss function of the model is:

[0048]

[0049] where represents the loss of the generative model, and P(g i |X, g1, g2,.., g i-1 ) represents the conditional probability of g i-1 when the input word embedding sequence X and the label word embeddings g1, g2,.., g i are known, and y i,j represents the logical value of the word j at position i.

[0050] The trained generative model can generate a text sequence of sentiment triples in a specified order according to the order hint in the input.

[0051] (3) Discriminative Model Input Embedding Module

[0052] The role of the discriminative model input embedding module is to convert the generation result of the generative model into the input of the discriminative task, so as to dynamically train the discriminative model with the generative model. Specifically, the module performs formatting, shuffling and reconstruction, and encoding operations on the output sequence of the generative model in turn.

[0053] ① Formatting: The module first formats the text sequence Op of sentiment triples generated by the generative model. Op is directly obtained by decoding the output embedding Y. The output of the trained generative model should have a similar format to its output label Go. According to the label "[SSEP]", Op is split into multiple sentiment triples, and the corresponding texts are extracted according to the labels "[A], [O], [S]" respectively, and converted into sentiment triples in a unified format.

[0054] ② Shuffle and reconstruction: Shuffle all the sentiment triples in unified format generated by the generation module from the same original input text St, as well as the formatted labeled sentiment triples. Shuffle the order of all the aggregated sentiment triples (including those generated by the generation model and the labeled ones) to prevent the discriminative model from learning irrelevant features. Concatenate each sentiment triple after the original text and separate them with the separator token "[SEP]" to form the reconstructed input statement, and set a new label for the reconstructed input statement according to the label content. The discriminative model processes a binary classification task, so the reconstructed input label is binary, where "1" represents that the current sentiment triple is a correct interpretation of the original text, and "0" represents that the current sentiment triple does not match the semantics of the original text.

[0055] ③ Encoding: Perform word embedding on the reconstructed input Sr, add the "[CLS]" token at the head of Sr, add position embedding and segment embedding, and finally obtain the input word embedding sequence X of the discriminative model. DIS :

[0056] X DIS = embedding BERT (Sr).

[0057] Where Sr represents the reconstructed input, and embedding BERT (·) represents the word embedding model of the pre-trained discriminative model.

[0058] The input embedding module of the discriminative model passes the word embedding sequence X DIS to the sentiment triple discriminative model as the input.

[0059] (4) Sentiment triple discriminative model

[0060] Adopt a fine-tuned open-source encoder-decoder structure model (such as the BERT-base model) as the sentiment triple discriminative model, and add a classification head at the top of the model to adapt to the discriminative task. The discriminative model takes the word embedding sequence of the reconstructed input text as the input, captures the global semantic features in the reconstructed input text through the multi-head self-attention mechanism, and discriminates whether the given sentiment triple is a correct interpretation of the original text. If the current sentiment triple is consistent with the semantics of the original text, it outputs 1, otherwise it outputs 0.

[0061] The input of the discriminative model is the original text sentence concatenated with the sentiment triple text, and the two texts are separated by "[SEP]". The output of the discriminative task is binary, which is similar to the pre-training task of judging whether it is the next sentence in the BERT pre-training model. The training objectives of both are to enable the model to recognize the relationship between the front and back sentences separated by "[SEP]". Therefore, the BERT pre-training model is selected as the core part of the discriminative model.

[0062] Add a classification head on top of the BERT model to enable the model to discriminate sentiment triples. The classification head is a fully connected layer that receives the hidden state of the "[CLS]" token at the head and outputs to a two-dimensional logical value z through a linear layer transformation. This process is shown by the following formula: DIS where z represents the final logical value of the discrimination model, h

[0063]

[0064] represents the hidden state of the "[CLS]" label of the discrimination model, W represents the weight matrix, b represents the bias vector, and d [CLS] represents the word embedding dimension of the discrimination model. B

[0065] The training loss function of the discrimination model is:

[0066]

[0067] where represents the loss of the discrimination model, represents the classification prediction probability corresponding to the discrimination model, and I represents the true classification indication label of the sample, taking 1 when it is true and 0 when it is false.

[0068] Train on the sentiment triple discrimination task to fine-tune the parameters. The sentiment triple is essentially a highly condensed generalization of the sentiment subject and sentiment expression of the original text. A correct sentiment triple should have the same semantics as the original text. The reconstructed input text may contain labeled sentiment triples, i.e., correct sentiment triples, or may contain incorrect sentiment triples generated by the generation model. The setting of the discrimination task can help the model learn to distinguish reasonable sentiment generalizations from those that are inconsistent with the expression of the original text, and achieve the recognition of errors generated by the generation model.

[0069] (5) Sentiment triple output module

[0070] The task of the sentiment triple output module is to integrate the outputs of the generation model and the discrimination model and form the final prediction result. Specifically, the output module first aggregates all the sentiment triples generated by the generation module for the same original text, and then screens and evaluates the quality of these triples through the discrimination model. For the sentiment triples judged to be incorrect and those with missing components (such as missing sentiment subject, opinion word, or sentiment tendency), the module removes them to ensure the integrity and accuracy of the output result. Finally, the sentiment triples judged to be correct by the discrimination model are retained as the final output set of sentiment triples.

[0071] The training method of the above model includes the following steps: ​

[0072] (1) Dataset preprocessing: Convert the dataset with indexed labels into a dataset with natural language labels. Before training the model, first convert the indexed labels in the dataset into corresponding natural language words. Then, paraphrase the sentiment polarity into natural language. Specifically, paraphrase the positive label "POS" as "great", the negative label "NEG" as "bad", and the neutral label "NEU" as "ok".

[0073] (2) Train the sentiment triple generation model: For each entry in the dataset, construct a full-view prompt with sentiment element tags "[A]", "[O]", and "[S]". There are six different full permutations of the three sentiment element tags, as shown below, that is, six views. Connect the original text with the prompts of the six views in turn, and add the task description prompt "ASTE:" at the head of the text to form six different inputs. Train the sentiment triple generation model by combining the six labels corresponding to the six views. The trained sentiment triple generation model will generate a set of sentiment triples in the corresponding order according to the prompt of each view. T in the following figure Figure 2 represents the set of sentiment triples generated by the i-th view. In the model training stage, retain all the sentiment triple sets, whether correct or incorrect, generated by all views for use as training samples for the sentiment triple discriminant model. i

[0074] (3) Use the generation results to train the discriminant model: For an entry in the dataset, collect the full-view sentiment triple set generated with it as the input, and pass it into the input embedding module of the discriminant model. This module formats, shuffles and reconstructs, and encodes the sentiment triples in the set to synthesize the training sample format of the discriminant model. The sentiment triple discriminant model is trained using these reconstructed samples as the training dataset.

[0075] (4) Alternating training: After the sentiment triple generation model has been trained for several training cycles, collect the intermediate results generated by the current generation model to train the sentiment triple discriminant model; after the sentiment triple discriminant model has been trained for several training cycles, it already has a certain ability to distinguish errors. Its discriminant results can be used through a feedback mechanism to guide the generation model to pay more attention to the training samples with generation errors, so as to generate higher-quality triples. Subsequently, the sentiment triple generation model resumes training based on the previous training parameters. After several training cycles of training, collect the intermediate results generated by the current generation model again to continue training the sentiment triple discriminant model. The two are trained alternately until the set target training rounds are reached. Through the interaction between the generation model and the discriminant model, the generation model can generate more accurate triples, and the discriminant model can effectively distinguish correct and incorrect triples, improving the overall accuracy of the model in the sentiment triple extraction task.​

[0076] During alternating training, during the implementation of alternating training, considering that after the sentiment triple generation model is fully trained on the training set, if the same training set is still used to generate the training samples of the discriminant model, it may lead to insufficient quantity and diversity of misexample samples, thus affecting the training effect of the sentiment triple discriminant model.

[0077] To avoid the above problems, the training set is divided into non-overlapping subsets A and B, where subset A contains data with a proportion of α in the original training set, and subset B contains the remaining 1 - α of the data.

[0078] The alternating training phase can be divided into three phases as shown in Figure 3 the following:

[0079] Phase I: At the initial stage of alternating training, subset A is used as the initial training data for the sentiment triple generation model. After training for θ1 iteration rounds, the training of the sentiment triple generation model is paused. The text in subset B is used as the input of the sentiment triple generation model, and the sentiment triples of the subset B text generated by the generation model are collected. The input embedding module of the discriminant model completes the matching of the generated sentiment triples with the true labels of subset B, and the generation results of the sentiment triple generation model are converted into the training samples of the sentiment triple discriminant model. The method of undersampling (negative sample quantity > positive sample quantity) or oversampling (negative sample quantity < positive sample quantity) is used to ensure the balance of positive and negative samples. The sentiment triple discriminant model is trained on the training data set transformed from subset B. After training for θ2 iteration rounds, the training is paused and phase II is entered.

[0080] Phase II: Based on the training in phase I, subset B is used as the training data set of the sentiment triple generation model, and the training is continued. The training process is similar to that in phase I. After training for θ1 iteration rounds, the training of the sentiment triple generation model is paused. The sentiment triple generation results of subset A generated by the generation model are collected, and the training samples of the discriminant model for subset A are synthesized to supplement the discriminant model training set. The sentiment triple discriminant model continues to resume training on the updated training data set based on the training in phase I. After training for θ2 iteration rounds, the training is paused and phase III is entered.

[0081] Phase III: Merge the data of subset A and subset B, that is, continue to train the sentiment triple generation model using all the data of the original training set on the basis of the training in Phase II. Stop training after θ1 iteration rounds, collect the sentiment triple generation results of the generation model for subset A + subset B, and synthesize the training samples for the discriminant model to supplement and update the discriminant model training set. The sentiment triple discriminant model continues to resume training using the updated training data set on the basis of the training in Phase II. Stop training after θ2 iteration rounds to obtain the final model.

[0082] Feedback mechanism adopted by the discriminant model:

[0083] During the alternating training process, the discriminant model has acquired a certain ability to distinguish between correct and incorrect by learning from the wrong examples generated by the generation model and the labeled sentiment triples (i.e., positive examples). At this time, the discriminant model can give feedback on the sample generation quality to the generation model based on its discrimination results, thereby guiding the generation model to focus on the samples that may generate generation errors.

[0084] Specifically, in Phase I of the alternating training, as described above, the sentiment triple discriminant model is trained using the generation results of the sentiment triple generation model for subset B. After training for a certain number of rounds, stop updating the parameters, and input the generation results of subset B into the discriminant model again to calculate the confidence score of the discriminant model for the samples in subset B. The formula for calculating the confidence Conf(t|St) of the sentiment triple t of the sample St in subset B by the discriminant model is as follows:

[0085]

[0086] Among them, Conf(t|St) represents the confidence score of the discriminant model for the sentiment triple t of the sample St, z represents the final logical value of the discriminant model, h [CLS] represents the hidden state of the [CLS] label of the discriminant model, W represents the weight matrix, b represents the bias vector, and d B represents the word embedding dimension of the discriminant model.

[0087] Collect the samples with confidence scores lower than the threshold β to form a low-confidence sample set:

[0088] U β ={(St,t)|Conf(t|St)<β}

[0089] Among them, U β represents the low-confidence sample set, St represents the original input text, t represents the sentiment triple, and β is the threshold.

[0090] In Phase II, the low-confidence sample set U βThe samples involved will be oversampled during the process of training the generative model. In Phase II, similar to Phase I, the sentiment triple discriminant model is trained using the updated training dataset based on the training in Phase I. After reaching the target number of iteration rounds, the training is paused, the parameters are stopped from being updated, and the updated training samples are input into the discriminant model again. The confidence score of the discriminant model for the samples is calculated, and the low-confidence sample set U is collected. β The samples involved in the low-confidence sample set U β will be oversampled during the process of training the generative model in Phase III.

[0091] It should be noted that the multi-head self-attention mechanism involved in the present invention is a prior art. The self-attention mechanism is a method in the Transformer model for capturing long-range dependencies in the input sequence. Its core idea is to determine which elements are more important for the current element by calculating the attention weights of each element in the input sequence for all other elements. In practical applications, the self-attention mechanism usually appears in the form of multiple heads. The output of the multi-head self-attention is to concatenate the outputs of all heads and then obtain the final output through a linear transformation.

[0092] In text sentiment analysis, using the self-attention mechanism can make the representation of each word in the output consider the global context information, rather than only relying on the information of a fixed window size, and can better capture long-range dependencies.

[0093] Based on the idea of first complex and then simple, the present invention guides the generative model to learn the semantic information in the label sentiment triples from different perspectives through the label order hint of the full view, and learns the generation error case features of the generative model through the discriminant model. The extraction results of the full view are discriminated and filtered to eliminate the sentiment triples with adjacent dependency mismatch problems, and at the same time avoid the generation errors caused by the convergence of multi-view predictions; by innovatively designing an alternating training and collaborative optimization scheme for the generative model and the discriminant model, the overall accuracy of the model for extracting sentiment triples is effectively improved, and a fine-grained solution can be provided for public opinion analysis.

Claims

1. A full-view prompt generation and misexample discrimination cross-training model for aspect-level sentiment triple extraction, characterized by: Including: A generation model input embedding module, which is used to convert the original data into a format suitable for the generation model; An emotion triple generation model, which uses a fine-tuned open-source encoder-decoder structure generation model as the generation model of emotion triples. The generation model takes the word embedding sequence obtained after being processed by the input embedding module as the input, and outputs a natural language-based emotion triple sequence that conforms to the sequential prompts in the input; A discriminant model input embedding module, which converts the generation result of the generation model into the input of the discrimination task to obtain the input word embedding sequence of the discriminant model; An emotion triple discriminant model, which uses a fine-tuned open-source encoder-decoder structure model as the discriminant model of emotion triples. A classification head is added to the top of the model to adapt to the discrimination task. The discriminant model takes the word embedding sequence obtained after being processed by the input embedding module as the input and outputs the classification of the discrimination result; An emotion triple output module, which combines the outputs of the emotion triple generation model and the emotion triple discriminant model, filters out the emotion triples judged to be incorrect, and forms the final prediction result.

2. The full-view hint generation and wrong example discrimination cross-training model for aspect-level sentiment triple extraction according to claim 1, characterized in that: In the generative model input embedding module, the text "ASTE:" is first added to the header of the original input data as a prompt. Then, a prompt containing a specific label sequence is constructed and the sequential prompt is cascaded to the end of the text to form the input text St of the generative model. The input text St is vectorized to form a generative model word embedding sequence X consisting of word units. GEN , and then embed the word into the sequence X GE Pass the sentiment triplet generation model as input.

3. The full-view hint generation and misexample discrimination cross-training model for aspect-level sentiment triple extraction according to claim 2, characterized in that: In the discriminant model input embedding module, the module needs to perform formatting, shuffling and reconstruction, and encoding operations on the output sequence of the generation model in sequence; during formatting, the emotional triple text sequence generated by the generation model is converted into an emotional triple in a unified format; during shuffling and reconstruction, first aggregate all the emotional triples in the unified format and the formatted labeled emotional triples generated by the generation model from the same original input text St, then shuffle the order of all the aggregated emotional triples, and then connect each emotional triple after the original text, separated by the separator marker "[SEP]" to form a reconstructed input statement, and set a new label for the reconstructed input statement according to the label content; during encoding, perform word embedding on the reconstructed input, denoted as Sr, add the "[CLS]" marker at the head of Sr, add position embedding and segment embedding, and finally obtain the input word embedding sequence X of the discriminant model DIS , the discriminant model input embedding module inputs the word embedding sequence X DIS into the emotional triple discriminant model as input.

4. The full-view prompt generation and misexample discrimination cross-training model for aspect-level sentiment triple extraction according to claim 3, characterized in that: The emotion triple discriminant model takes the word embedding sequence of the reconstructed input text as the input, captures the global semantic features in the reconstructed input text through the multi-head self-attention mechanism, and discriminates whether the given emotion triple is a correct extraction of the original text.

5. The full-view hint generation and misexample discrimination cross-training model for aspect-level sentiment triple extraction according to claim 4, characterized in that: In the emotion triple output module, first, all the emotion triples generated by the generation module for the same original text are summarized, and then these triples are screened and evaluated by the discriminant model; for the emotion triples judged to be incorrect and the emotion triples with missing constituent elements, this module eliminates them. Finally, the emotion triples judged to be correct by the discriminant model are retained as the set of emotion triples for the final output.

6. The training method of the model according to any one of claims 1-5, characterized in that: Including the following steps: (1) Dataset preprocessing: Before training the model, first convert the index-type labels in the dataset into corresponding natural language words, and then paraphrase the sentiment polarity into natural language; (2) Training the emotion triple generation model: For each entry in the dataset, construct a full-view prompt with emotion element tags "[A]", "[O]", "[S]"; sequentially connect the original text with the prompts of the six views of the emotion element tags, and add a task description prompt "ASTE:" at the head of the text to form six different inputs. Combine the six labels corresponding to the six views to train the emotion triple generation model. The trained emotion triple generation model generates a set of emotion triples containing the corresponding sequential emotion triples according to the prompts of each view; (3) Using the generation result to train the discriminant model: For an entry in the dataset, collect the set of full-view emotion triples generated with it as the input, and pass it into the discriminant model input embedding module. This module formats, shuffles and reconstructs, and encodes the emotion triples in the set to synthesize the training sample format of the discriminant model. The emotion triple discriminant model uses these reconstructed samples as the training dataset for training; (4) Alternating training: After the emotion triple generation model has been trained through multiple training cycles, collect the intermediate results generated by the current generation model to train the emotion triple discriminant model; After the emotional triple discriminant model is trained through multiple training cycles, according to its discriminant results, it uses a feedback mechanism to reversely guide the emotional triple generation model to focus on the training samples with generation errors, so as to generate higher-quality triples; the two are alternately trained until the set target training rounds are reached.

7. The training method of the model according to claim 6, characterized in that: During alternating training, the training set is divided into non-overlapping subsets A and B, where subset A contains data with a proportion of α in the original training set, and subset B contains the remaining 1 - α of the data; The alternating training stage includes the following three stages: Stage I: In the initial stage of alternating training, use subset A as the initial training data for the emotional triple generation model; after training for θ1 iteration rounds, pause the training of the emotional triple generation model, use the text in subset B as the input of the emotional triple generation model, collect the emotional triples of the generation model for the text in subset B, and complete the matching of the generated emotional triples with the true labels of subset B by the discriminant model input embedding module, and convert the generation results of the emotional triple generation model into training samples for the emotional triple discriminant model; The emotional triple discriminant model is trained on the training data set transformed from subset B, and after θ2 iteration rounds, pause the training and enter Stage II; Stage II: On the basis of the training in Stage I, use subset B as the training data set for the emotional triple generation model and continue to start training; after training for θ1 iteration rounds, pause the training of the emotional triple generation model, collect the generation results of the emotional triples of the generation model for subset A, and synthesize the training samples for the discriminant model for subset A to supplement the discriminant model training set; The emotional triple discriminant model continues to resume training on the basis of the training in Stage I using the updated training data set, and after θ2 iteration rounds, pause the training and enter Stage III; Stage III: Combine the data of subset A and subset B, that is, continue to train the emotional triple generation model on the basis of the training in Stage II using all the data of the original training set. After training for θ1 iteration rounds, stop the training, collect the generation results of the emotional triples of the generation model for subset A and subset B, and synthesize the discriminant model training samples to supplement and update the discriminant model training set; The emotional triple discriminant model continues to resume training on the basis of the training in Stage II using the updated training data set, and after θ2 iteration rounds, stop the training to obtain the final model.

8. The training method of the model according to claim 7, wherein: the feedback mechanism adopted by the discriminant model is that the discriminant model calculates the confidence score of the sample according to the following formula during the training process, collects the samples with the confidence score lower than the threshold β to form a low-confidence sample set, and the samples involved in the low-confidence sample set are oversampled during the training process of the generation model in the next alternating training round; Among them, Conf(t|St) represents the confidence score of the discriminative model for the sentiment triple t of the sample St, z represents the final logical value of the discriminative model, and h [CLS] represents the hidden state of the [CLS] label of the discriminative model, W represents the weight matrix, b represents the bias vector, and d B represents the word embedding dimension of the discriminative model.

Images