Attribute-level emotion triple extraction method based on pairing information enhancement

Through the attribute-level affective triple generation model based on the encoder-decoder structure, combined with cross-entropy loss and contrast learning methods, the error propagation, insufficient pairing learning and limitations in the attribute-level affective triple extraction are solved. , improves the extraction effect and robustness.

CN118916446AInactive Publication Date: 2024-11-08SUZHOU UNIV

Patent Information

Application Number
CN202411390313.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-08
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing attribute-level emotional triple extraction methods have problems such as error propagation, large computing overhead, complex modeling and reasoning, insufficient pairing learning of attribute-opinion terms, and limited emotional judgment ability.

Method used

The baseline model is generated using an attribute-level emotional triplet based on the encoder-decoder structure, and the comment text encoding and triplet generation are used to use the T5 encoder and T5 decoder, and the cross entropy loss function, InfoNCE loss function and comparison learning are used. Methods: Multi-task joint training is carried out to enhance the matching and emotional judgment ability of attribute words and opinion words.

Benefits of technology

It significantly improves the matching accuracy of attribute words and opinion words and the accuracy of emotional judgment, avoids error propagation problems, and improves the overall quality and robustness of triple extraction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916446A_ABST
    Figure CN118916446A_ABST
Patent Text Reader

Abstract

The invention discloses an attribute-level emotion triple extraction method based on pairing information enhancement, and relates to the technical field of natural language processing, and the method comprises the steps: obtaining a comment text data set containing user evaluation; designing an attribute-level emotion triple based on an encoder-decoder structure to generate a baseline model; a T5 encoder is used for encoding the comment text into hidden representation, and an attribute-level emotion triple is generated through a T5 decoder; calculating triple generation loss by adopting a cross entropy loss function; multi-level matching information enhancement is executed, and matching layer comparison loss and emotion layer comparison loss are calculated through comparison learning of a matching layer and an emotion layer; and combining the triple generation loss, the pairing layer comparison loss and the emotion layer comparison loss to construct a comprehensive loss function and perform multi-task joint training. According to the method, the T5 model is adopted, the error propagation problem in a traditional non-generative method is avoided, meanwhile, label semantics can be explored more deeply, and the overall quality of triple extraction is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to an attribute-level sentiment triple extraction method based on pairing information enhancement. Background Art

[0002] There are several major shortcomings in existing research on the attribute-level sentiment triple extraction task. First, the pipeline approach adopted in the early days is prone to error propagation and increased computational overhead, which affects the accuracy and processing efficiency of the extraction results. Although later methods such as grid annotation and location-aware annotation can extract all triplets at once, these methods often require complex modeling and reasoning processes. Second, although the research based on the encoder-decoder paradigm utilizes advanced pre-trained language models, it is still insufficient in the paired learning of attribute-opinion term pairs, and often results in mismatches or missed matches.

[0003] In addition, existing models also have limitations in their ability to determine sentiment. Due to the uneven distribution of sentiment samples in the corpus and the implicitness of some sentiment expressions, current triple extraction models perform poorly in identifying subtle sentiment differences between different attribute items in the text. These problems limit the application effect of triple extraction tasks, making it difficult for existing technologies to meet actual needs. Summary of the invention

[0004] In view of the problems of error propagation, high computational overhead, complex modeling and reasoning, insufficient learning of attribute-opinion term pairings, and limited sentiment judgment capabilities in existing attribute-level sentiment triple extraction methods, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is how to alleviate the error propagation brought by the pipeline method and the complex modeling problem brought by the unified labeling framework, as well as how to solve the mismatching problem between attribute words and opinion words and the error problem in judging the sentiment tendency in the process of generating attribute-level sentiment triples.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, an embodiment of the present invention provides an attribute-level sentiment triple extraction method based on pairing information enhancement, which includes obtaining a review text dataset containing user evaluations; designing an attribute-level sentiment triple generation baseline model based on an encoder-decoder structure, wherein the attribute-level sentiment triple generation baseline model includes a T5 encoder and a T5 decoder; using the T5 encoder to encode the review text into a hidden representation, and based on the hidden representation, generating an attribute-level sentiment triple through the T5 decoder, wherein the attribute-level sentiment triple includes two output paradigms, an annotation type and an extraction type; and using a cross entropy loss function to calculate the triple generation loss. ; Design the description information of term pairing knowledge and sentiment judgment knowledge, extract the dense vector representation of attribute-opinion pairs from the encoder output of the attribute-level sentiment triple generation baseline model, and perform multi-level pairing information enhancement; use the InfoNCE loss function to calculate the pairing layer contrast loss Compared with the sentiment layer loss ; Combined with triple generation loss , paired layer contrast loss Compared with the sentiment layer loss , construct a comprehensive loss function , perform multi-task joint training; use the trained attribute-level sentiment triples to generate a baseline model and extract attribute-level sentiment triplets from new review texts.

[0007] As a preferred solution of the attribute-level sentiment triple extraction method based on pairing information enhancement described in the present invention, the specific formula of the T5 encoder is as follows: ; in, is the output hidden representation of the T5 encoder, d is the dimension of the hidden representation, For T5 encoder, is the word sequence of the input comment text, n is the sequence length, and R is a real number set.

[0008] The specific formula of the T5 decoder is as follows: ; in, is the output of the T5 decoder at time step t, d is the dimension of the hidden representation, For T5 decoder, and are the learnable parameter matrices and bias vectors, for Activation function, is the output sequence of the decoder before time step t, is the vocabulary probability distribution at the current time step t, is the output hidden representation of the T5 encoder, R is a real number set, and n is the sequence length.

[0009] As a preferred solution of the attribute-level sentiment triple extraction method based on pairing information enhancement described in the present invention, wherein: multi-level pairing information enhancement includes pairing layer information enhancement and sentiment layer information enhancement, and the execution steps of the pairing layer information enhancement are as follows: define a description information set, the description information set includes a matching type description and a non-matching type description; encode the pairing description information based on the BERT model to obtain the description information representation; extract the hidden layer representation of attribute words and opinion words from the output of the T5 encoder, randomly splice the hidden layer representation of attribute words and opinion words, and obtain the term pair representation through the linear layer; use the contrastive learning method to calculate the similarity between the description information representation and the term pair representation; use the pairing layer contrast loss function to optimize the parameters of the attribute-level sentiment triple generation baseline model, promote the model to extract valuable features of the pairing, so as to enhance the matching of attribute words and opinion words.

[0010] As a preferred solution of the attribute-level sentiment triple extraction method based on pairing information enhancement described in the present invention, the specific formula for obtaining the description information representation is as follows: ; in, for The output of the position, and is a learnable parameter, is the pairing information description, m represents the number of words, is the pre-trained bidirectional Transformer encoder function, is the activation function, The length of this sequence unit for description information.

[0011] The specific formula for term pair representation is as follows: ; in, and are the dense representations of attribute words and opinion words, respectively. is an average pooling function, and is a learnable parameter, Comment text The position index of attribute words and opinion words in , is the output hidden representation of the T5 encoder, is the term pair representation, are the starting position and ending position index of the attribute word respectively. are the starting and ending position indexes of the opinion word, respectively. This is a vector concatenation operation.

[0012] As a preferred solution of the attribute-level sentiment triple extraction method based on pairing information enhancement described in the present invention, the execution steps of sentiment layer information enhancement are as follows: define a sentiment category set, the sentiment category set includes three sentiment categories: positive, neutral and negative; encode the sentiment category description based on the BERT model, and use the new linear layer to obtain the sentiment information representation; extract the hidden layer representation of the real attribute word and opinion word pair from the output of the T5 encoder; convert the hidden layer representation of the real term pair into the term pair representation through the linear layer; use the contrastive learning method to calculate the similarity between the sentiment information representation and the term pair representation; optimize the parameters of the attribute-level sentiment triple generation baseline model based on the sentiment layer contrast loss function to improve its understanding and distinction ability of different sentiments; sentiment information representation The specific formula is as follows: ; in, is the activation function, for The output of the position, and are learnable parameters.

[0013] As a preferred solution of the attribute-level sentiment triple extraction method based on pairing information enhancement described in the present invention, the sentiment layer contrast loss The specific formula is as follows: ; in, is the set of representations for all true term pairs that do not belong to the current sentiment category, To represent emotional information, is the real term pair information in the sequence, is the temperature coefficient, For real term information and the similarity of emotional information representation.

[0014] As a preferred solution of the attribute-level sentiment triple extraction method based on pairing information enhancement described in the present invention, the comprehensive loss function The specific formula is as follows: ; in, is the weight hyperparameter, They are triplet generation loss, pairing layer contrast loss, and sentiment layer contrast loss respectively.

[0015] In the second aspect, an embodiment of the present invention provides an attribute-level sentiment triple extraction system based on pairing information enhancement, which includes a text data acquisition module for acquiring a review text dataset containing user evaluations; a baseline model construction module for constructing an attribute-level sentiment triple generation baseline model based on an encoder-decoder structure; a triple generation module for encoding the comment text into a hidden representation using a T5 encoder, and based on the hidden representation, generating an attribute-level sentiment triple through a T5 decoder, wherein the attribute-level sentiment triple includes two output paradigms, an annotation type and an extraction type; and a loss calculation module for calculating the triple generation loss. , paired layer contrast loss Compared with the sentiment layer loss ; Multi-level pairing information enhancement module, used to design the description information of term pairing knowledge and sentiment judgment knowledge, extract the dense vector representation of attribute-opinion pairs from the encoder output of the triple generation model, and perform multi-level pairing information enhancement; Multi-task joint training module combined with triple generation loss , paired layer contrast loss Compared with the sentiment layer loss , construct a comprehensive loss function , and conduct multi-task joint training; the attribute-level sentiment triplet generation module is used to use the trained attribute-level sentiment triplet to generate a baseline model and extract attribute-level sentiment triplet from new review text.

[0016] In a third aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program instructions are executed by the processor, the steps of the attribute-level sentiment triple extraction method based on pairing information enhancement as described in the first aspect of the present invention are implemented.

[0017] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program instructions are executed by a processor, the steps of the attribute-level sentiment triple extraction method based on pairing information enhancement as described in the first aspect of the present invention are implemented.

[0018] The beneficial effects of the present invention are as follows: the present invention enhances the applicability of the model by designing two output paradigms, namely annotation and extraction. Through comparative learning of the pairing layer and the sentiment layer, the present invention significantly improves the matching accuracy of attribute words and opinion words, as well as the accuracy of sentiment judgment. By adopting a generative model based on T5, the present invention avoids the error propagation problem in traditional non-generative methods, and at the same time can explore label semantics more deeply, improving the overall quality of triple extraction. By combining triple generation loss, pairing layer contrast loss and sentiment layer contrast loss, the present invention realizes multi-task joint training. This method not only improves the overall performance of the model, but also enhances the robustness of the model when dealing with complex emotional expressions. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 The framework flow chart of the attribute-level sentiment triple extraction method based on pairing information enhancement.

[0021] Figure 2 This is the model architecture diagram of the attribute-level sentiment triple extraction method based on pairing information enhancement.

[0022] Figure 3 A testing framework for attribute-level sentiment triple extraction methods based on pairing information enhancement.

[0023] Figure 4 This is the system module diagram of the attribute-level sentiment triple extraction method based on pairing information enhancement.

[0024] Figure 5 Computer device diagram for attribute-level sentiment triple extraction method based on pairing information enhancement. DETAILED DESCRIPTION

[0025] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0026] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0027] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0028] Example 1

[0029] Reference Figure 1~Figure 5 , which is the first embodiment of the present invention, and provides an attribute-level sentiment triple extraction method based on pairing information enhancement, comprising: S1: Get a review text dataset containing user reviews.

[0030] Specifically, the framework flow chart of the present invention is as follows Figure 1 As shown in Figure 2, the overall model architecture is as follows: Figure 2 As shown in the figure, two types of description information are designed, one to indicate whether the attribute words and opinion words match, and the other to indicate the meaning of the three tendencies of positive, negative and neutral emotions. The two types of description information are encoded using an encoder independent of the triple extraction model. Subsequently, the dense vector representations of the attribute words and opinion words are extracted from the encoder output of the triple extraction model and randomly paired. Based on the pairing information embedding and the random term pair embedding, a contrastive learning strategy is adopted to enhance the model's ability to match attribute words and corresponding opinion words. At the same time, contrastive learning is applied between the emotional information embedding and the correct term pair embedding to enhance the model's understanding of the emotional representation of the term pair and improve the emotional judgment ability.

[0031] S2: Design an attribute-level sentiment triplet generation baseline model based on the encoder-decoder structure (i.e., T5 model).

[0032] It should be noted that traditional non-generative methods usually use task-specific classification networks to model and predict terminology and sentiment tendencies, and use category indexes as labels. On the one hand, this discriminative modeling method may bring about error propagation problems. On the other hand, the modeling is complex and usually ignores the rich semantics within the label. In contrast, the generative model can explore label semantics more deeply by gradually decoding sequences containing attribute words, opinion words, and sentiment tendencies. Based on this, the present invention uses T5 as the baseline model. T5 is an encoder-decoder based model that is pre-trained on a mixture of multiple unsupervised and supervised task datasets, and can solve triple extraction tasks in a generative manner.

[0033] S3: The review text is encoded into a hidden representation using the T5 encoder, and attribute-level sentiment triples are generated based on the hidden representation using the T5 decoder.

[0034] First, the review text X is encoded into a hidden representation using the T5 encoder , the specific formula is as follows: , in, is the output hidden representation of the T5 encoder, d is the dimension of the hidden representation, For T5 encoder, is the word sequence of the input comment text, n is the sequence length, and R is a real number set.

[0035] It should be noted that, referring to relevant research experience, the present invention divides the output paradigm of the T5 model into annotation and extraction. The annotation output labels each attribute word with its corresponding opinion word and sentiment polarity to form a structure of [attribute word | opinion word | sentiment tendency]. This method intuitively presents the relationship between attributes, opinions, and sentiments, which is easy to understand and analyze. The extraction output adopts the concatenation and combination of triplets. Specifically, the model directly generates a complete triple sequence consisting of attribute words, opinion words, and sentiment polarity. This method is closer to the final application format, which is conducive to subsequent processing and utilization.

[0036] Example, comment text: Nice keyboard, battery and screen work ok (translation: The keyboard is nice, the battery and the screen are also OK); generated goals (annotation style): Nice [keyboard|positive|Nice], [battery|neutral|ok] and [screen|neutral|ok] work ok (translation: Nice, [keyboard|positive|Nice], [battery|neutral|ok] and [screen|neutral|ok] work ok); generated goals (extraction style): (keyborad, Nice, positive); (battery, ok, neutral); (screen, ok, neutral) (translation: (keyboard, nice, positive); (battery, ok, neutral); (screen, ok, neutral)).

[0037] Furthermore, in order to generate the output defined above, the T5 decoder outputs and the decoder output of the previous time step As input, to get the decoder output of the current step , then a fully connected softmax layer will Mapped to the vocabulary distribution, the word probability is generated. The specific formula is as follows: , in, is the output of the T5 decoder at time step t, d is the dimension of the hidden representation, For T5 decoder, and are the learnable parameter matrices and bias vectors, for Activation function, is the output sequence of the decoder before time step t, is the vocabulary probability distribution at the current time step t, is the output hidden representation of the T5 encoder, R is a real number set, and n is the sequence length.

[0038] S4: Use the cross entropy loss function to calculate the triple generation loss .

[0039] Furthermore, for the above triple generation, the cross entropy function is used to calculate the triple generation loss , the specific formula is as follows: , in, is the correct tag, Represents attribute words, opinion words, sentiment labels, or other content in the target output.

[0040] S5: Perform multi-level pairing information enhancement and calculate the pairing layer contrast loss using contrastive learning of the pairing layer and the sentiment layer Compared with the sentiment layer loss .

[0041] Specifically, the multi-level pairing information enhancement includes pairing layer information enhancement and sentiment layer information enhancement, including the following steps: S5.1: Perform paired layer information enhancement.

[0042] It should be noted that the present invention defines As a set of descriptive information indicating whether the attribute words and opinion words match, the specific definitions are as follows: Pair (match): The tokens have an emotional connection. (Translation: These words have emotional connection); Unpair (mismatch): The tokens do not have any emotional connection. (Translation: These words do not have emotional connection).

[0043] Specifically, a description information set is defined, wherein the description information set includes a matching type description and a non-matching type description; based on the BERT model, a pairing information description containing m words is Encode and obtain the description information representation. The specific formula is as follows: , in, for The output of the position, and is a learnable parameter, is the pairing information description, m represents the number of words, is the pre-trained bidirectional Transformer encoder function, is the activation function of the linear rectifier unit with leakage, The length of this sequence unit for description information.

[0044] Furthermore, in order to generate correct paired and mismatched sample instances, the output of the T5 encoder Extract the hidden representation of attribute words and opinion words, and randomly concatenate attribute words and opinion words The hidden layer representation is obtained through the linear layer to obtain the term pair representation. The specific formula is as follows: , in, and are the dense representations of attribute words and opinion words, respectively. is an average pooling function (mapping n output vectors into a representative vector), and is a learnable parameter, Comment text The position index of attribute words and opinion words in , is the output hidden representation of the T5 encoder, is the term pair representation, are the starting position and ending position index of the attribute word respectively. are the starting and ending position indexes of the opinion word respectively, This is a vector concatenation operation.

[0045] It should be noted that due to random coding, It may be a true match or a false match.

[0046] Furthermore, the test framework is Figure 3As shown. A contrastive learning method is used to calculate the similarity between the description information representation and the term pair representation; the parameters of the attribute-level sentiment triple generation baseline model are optimized based on the pairing layer contrast loss function to improve the matching of attribute words and opinion words. Contrastive learning ensures that the similarity between the hidden layer representation of the corresponding description information and the correct attribute-opinion word pair is maximized, while the similarity with the mismatched term pair is minimized. This process is achieved through the pairing layer contrast loss. accomplish: , in, represents the set of all term pairs that do not belong to the current pairing type, Description of pairing information. is the temperature coefficient, is the term pair representation, for The samples in .

[0047] Preferably, since the description information represents the center point of correct pairing and mismatching, it contains rich semantic information related to pairing. Through contrastive learning, the T5 model can extract pairs consisting of attributes and opinions and learn this information, thereby facilitating the extraction of valuable features for pairing. Therefore, this makes it possible to generate more accurate triples.

[0048] It should be noted that information enhancement at the pairing layer considers the pairing of attributes and opinions, and the model needs to learn whether a and b are related. Information enhancement at the sentiment layer is for related a and b, and the model needs to learn what the sentiment type of the two is.

[0049] S5.2: Perform sentiment layer information enhancement.

[0050] Specifically, the present invention defines As a specific explanation of the three labels of positive emotions, negative emotions and neutral emotions, the definitions are as follows: Pos (positive): Feelings of happiness, joy, orsatisfaction. (Translation: feelings of happiness, joy or satisfaction); Neu (neutral): Lack of significantemotional response. (Translation: lack of obvious emotional response); Neg (negative): Feelings of sadness, anger, or disappointments. (Translation: feelings of sadness, anger or disappointment).

[0051] Preferably, a set of sentiment categories is defined, the set of sentiment categories including three sentiment categories: positive, neutral and negative; the sentiment category description is encoded based on the BERT model, and the sentiment information representation is obtained using a new linear layer; the hidden layer representation of the real attribute word and opinion word pair is extracted from the output of the T5 encoder; the hidden layer representation of the real term pair is converted into a term pair representation through a linear layer; the similarity between the sentiment information representation and the term pair representation is calculated using a contrastive learning method; and the parameters of the attribute-level sentiment triple generation baseline model are optimized based on the sentiment layer contrast loss function to improve its ability to understand and distinguish different sentiments.

[0052] It should be noted that emotional information representation The specific formula is as follows: , in, It is a linear rectifier unit activation function with leakage, which solves the problem that the gradient of the ReLU function is zero in the negative region. It is a linear rectifier unit activation function with leakage. for The output of the position, and are learnable parameters.

[0053] Furthermore, in order to learn how to correctly pair attribute words and opinion words, random term pair encoding is used; in the present invention, in order to learn how to determine the emotional tendency expressed by the term pair, only the real term pair information in the sequence is extracted. Because the randomly paired term pairs contain combinations without actual emotional information, the model will learn incorrect emotional associations.

[0054] Specifically, through contrastive learning, the model can narrow the distance between the sentiment label and the corresponding term pair, thereby enhancing its ability to understand and distinguish different sentiments. This process is achieved through the pairing layer contrast loss function The specific formula is as follows: , in, is the set of representations for all true term pairs that do not belong to the current sentiment category, To represent emotional information, is the real term pair information in the sequence, is the temperature coefficient, For real term information and the similarity of emotional information representation.

[0055] Furthermore, through contrastive learning of sentiment views, the model can learn representations of three different sentiments: positive, negative, and neutral, to better understand the sentiment characteristics of term pairs and make more accurate predictions in sentiment determination tasks.

[0056] S6: Combine triplet generation loss, pairing layer contrast loss and sentiment layer contrast loss to construct a comprehensive loss function and perform multi-task joint training.

[0057] The comprehensive loss function The specific formula is as follows: , in, is the weight hyperparameter, which is set to 0.8, 0.15, and 0.05 in the present invention. Generate loss function for triples, contrast loss function for pairing layer and contrast loss function for sentiment layer respectively.

[0058] S7: Use the trained attribute-level sentiment triples to generate a baseline model to extract attribute-level sentiment triplets from new review texts.

[0059] It should be noted that the present invention is based on the generative sentiment triple generation model training process, which, on the one hand, solves the problem of mismatching between attribute words and opinion words in the attribute-level sentiment triple generation process, and on the other hand, solves the problem of sentiment judgment errors caused by incorrect term pairing.

[0060] Furthermore, this embodiment also provides an attribute-level sentiment triple extraction system based on pairing information enhancement, such as Figure 4 As shown, it includes a text data acquisition module for acquiring a review text dataset containing user evaluations; a baseline model construction module for constructing an attribute-level sentiment triple generation baseline model based on an encoder-decoder structure; a triple generation module for encoding the review text into a hidden representation using a T5 encoder, and generating an attribute-level sentiment triple through a T5 decoder based on the hidden representation, wherein the attribute-level sentiment triple includes two output paradigms, an annotation type and an extractive type; and a loss calculation module for calculating the triple generation loss. , paired layer contrast loss Compared with the sentiment layer loss ; Multi-level pairing information enhancement module, used to design the description information of term pairing knowledge and sentiment judgment knowledge, extract the dense vector representation of attribute-opinion pairs from the encoder output of the triple generation model, and perform multi-level pairing information enhancement; Multi-task joint training module, used to combine the triple generation loss , paired layer contrast loss Compared with the sentiment layer loss , construct a comprehensive loss function , and conduct multi-task joint training; the attribute-level sentiment triplet generation module is used to use the trained attribute-level sentiment triplet to generate a baseline model and extract attribute-level sentiment triplet from new review text.

[0061] This embodiment also provides a computer device, which is suitable for the case of an attribute-level sentiment triple extraction method based on pairing information enhancement, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the attribute-level sentiment triple extraction method based on pairing information enhancement proposed in the above embodiment.

[0062] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc. Figure 5 shown.

[0063] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for extracting attribute-level sentiment triples based on pairing information enhancement as proposed in the above embodiment is implemented.

[0064] In summary, the present invention enhances the applicability of the model by designing two output paradigms, annotation and extraction. Through comparative learning of the pairing layer and the sentiment layer, the present invention significantly improves the matching accuracy of attribute words and opinion words, as well as the accuracy of sentiment judgment. By adopting a generative model based on T5, the present invention avoids the error propagation problem in traditional non-generative methods, and at the same time can explore label semantics more deeply, improving the overall quality of triple extraction. By combining triple generation loss, pairing layer contrast loss and sentiment layer contrast loss, the present invention realizes multi-task joint training. This method not only improves the overall performance of the model, but also enhances the robustness of the model when dealing with complex emotional expressions.

[0065] Example 2

[0066] Reference Figure 1~Figure 5 , which is the second embodiment of the present invention, and this embodiment provides an attribute-level sentiment triple extraction method based on pairing information enhancement. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0067] Specifically, the present invention conducts experiments based on the public dataset ASTE-Data-V2 for attribute-level sentiment triple extraction, which can be used to make fair comparisons and evaluations with existing triple extraction related works. The data distribution is shown in Table 1.

[0068] Table 1 Data distribution

[0069] T5-base is used as the baseline model for triple generation to solve the problem that the discriminative model cannot fully utilize the semantic information of the label. BERT-base-cased is used for the description encoder. The output size of the linear layer is set to 128, and the temperature parameter of contrastive learning is set to 0.07. For the four data sets of 14lap, 14res, 15res and 16res, the learning rates are set to 4e-4, 2e-4, 2e-4 and 3e-4 respectively. In order to enhance the generalization ability of the model, the dropout rate is set to 0.1, and the data size of each batch is set to 16. The model training rounds are set to 20 to ensure that the model can fully learn the features in the data set. In the design of the loss function, different weights are set for different components. They are set to 0.8, 0.15 and 0.05 respectively. To ensure the stability of the evaluation results, the present invention reports the average F1 score of five random runs.

[0070] Furthermore, the comparison objects are as follows: Peng-two-stage (2020): This method is a two-stage triple extraction framework, which first extracts attribute-sentiment pairs and opinion words in the first stage, and then performs Cartesian product pairing and classification in the second stage. JET-BERT (2020): This method designs a unified annotation mode based on position awareness. GTS-BERT (2020): This method annotates the relationship between all word pairs in a tabular form, and designs an inference strategy to exploit the mutual indication between different opinion factors for more accurate extraction. S 3 E 2(2021): This method optimizes the GTS framework and represents the semantic and syntactic relationships between word pairs through graph neural networks to enhance the triple extraction performance. EMC-GCN (2022): This method converts the text into a multi-channel graph, considers multiple features such as part of speech, dependency, tree-based distance and relative position distance to enhance the graph model, and introduces 10 inter-word relationship labels. BMRC (2021): This method converts the attribute-level sentiment triple extraction task into a multi-round machine reading comprehension task, and sets multiple rounds of query questions from the two directions of attribute words and opinion words. COM-MRC (2022): This method introduces a text enhancement strategy of masking attribute words on the machine reading comprehension framework to avoid interference between different attribute words. ASTE-RL (2021): This method uses a hierarchical reinforcement learning framework to first identify the sentiment information of the comment text, and then searches for the target attribute words and opinion words corresponding to the sentiment. SpanCopy-RoBERTa (2022): This method introduces span copying and dual decoder modules to decode attribute words and opinion words respectively, and introduces information such as word meaning, part of speech, boundary labels, etc. on the encoder side. BART-ABSA (2021): This method uses BART as a generative model to convert the attribute-level sentiment triple extraction task into an index generation problem, where the term word index corresponds to the position in the sequence. GAS-annotation (2021): This method uses the T5 model to generate triples and introduces an annotation-style generation paradigm, in which each attribute term and its corresponding sentiment triplet are enclosed in brackets. GAS-extraction (2021): A variant of GAS-a, which concatenates all extractive-style triplets as output. ChatGPT (gpt-3.5-turbo) (2022): A large language model that accepts natural language sequence input and outputs specified content based on the information in Prompt.

[0071] Furthermore, the training set, validation set, and test set are divided from the ASTE-Data-V2 dataset to ensure that each subset contains enough sentences and sentiment category distribution. T5-base is loaded as the baseline model, and BERT-base-cased is configured as the description encoder. Parameters such as linear layer output size, contrast learning temperature parameter, learning rate, Dropout rate, batch size, and training rounds are set. The model is trained using the training set data, and the model performance is optimized by adjusting parameters such as learning rate and Dropout rate. After each round of training, the model is evaluated on the validation set, and the model parameters are adjusted or the training is stopped early according to the evaluation results to prevent overfitting. The training process is repeated until the preset training rounds are reached or other stopping conditions are met. The performance of the model is evaluated on the test set, and the average F1 score of five random runs is calculated and reported to ensure the stability of the evaluation results. Based on the experimental results, the advantages and disadvantages of the model in the attribute-level sentiment triple extraction task are analyzed, and possible improvement directions are explored.

[0072] The experimental results are shown in Table 2, in which HiTCPE (Hierarchical Triplet ContrastivePairing Enhancement) corresponds to the method proposed in the present invention.

[0073] Table 2 Experimental results

[0074] Preferably, among all the baselines compared, this method performs well, with HiTCPE-annotation and HiTCPE-extraction achieving the highest (68.10%) and second highest (67.66%) average F1 values, respectively. Among them, in the extractive output style, compared with GAS-extraction, this method improves the F1 value by 1.51%, 0.5%, 1.96% and 1.54% on the four datasets of 14lap, 14res, 15res and 16res, respectively. Similarly, when the output style is annotation, compared with GAS-annotation, this method improves by 6.22%, 3.43%, 5.65% and 3.8% on the four datasets, respectively. These results show that enhancing pairing information is crucial for ASTE, while the naive T5 generation process does not consider the pairing of control attribute words and opinion words. The introduction of contrastive learning enables the model to learn this information. In addition, the three restaurant datasets have more obvious improvements on HiTCPE-annotation, while 14lap has more obvious performance improvements on HiTCPE-extraction. This is because the output goal of the former is to require the model to use the three symbols "[", "]" and "|" to determine the triple range while generating the original sequence. This output has the information of the original sequence as a reference and only needs to be labeled with the correct range. The latter requires the model to generate more regularized triple information. In this case, for the dataset in the restaurant field, since its attribute words include a variety of food names and various raw materials as modifiers, the output paradigm of HITCPE-annotation has better performance because it has the original sequence output as the basis for triple annotation. For the 14lap dataset in the notebook field, HITCPE-extraction performs better due to the low diversity of its terminology.

[0075] Moreover, ChatGPT's triple extraction performance is poor whether it is without examples (zero-shot) or with examples (3-shot and 5-shot). This is mainly due to the particularity of ASTE, which belongs to the category of fine-grained sentiment analysis. This task requires the model to not only judge the emotional tendency of the sentence, but also accurately extract the attribute words and opinion words in it, and ensure the correct word boundaries. The training goal of the large model is mainly focused on generating natural and fluent text, rather than focusing on structured triple extraction in a specific domain. Therefore, ASTE is a challenging task for large models. In addition, it is worth noting that in the few-shot setting, the results of 5-shot and 3-shot are significantly better than zero-shot, which shows that it is necessary to provide examples for contextual learning for large models. This approach helps the model better understand the task requirements and learn extraction specifications.

[0076] For example, invalid pairing / invalid opinion words: In the first example, the opinion word "crispiness" is used to describe "batter", but GAS mistakenly pairs it with "prawns". In the second example, although GAS correctly extracts the two attribute words "runs" and "fans", it also mistakenly includes the non-opinion word "huge bonus" in the pairing category, which leads to inaccurate matching relationships and incorrect triples. The first two examples show that compared with the GAS method based only on the T5 model, the method proposed in this chapter produces fewer pairing errors and better triple extraction performance in complex contexts where sentences contain multiple attribute words and opinion words.

[0077] For example, sentiment judgment error: In the third example, GAS failed to identify the sentiment in the text that suggests that the employee is not currently friendly enough, and instead misjudged the sentiment of the triple as positive based on "friendly". In contrast, the method proposed in this chapter introduces a module for understanding the sentiment judgment of terms, which effectively avoids such errors.

[0078] In the fourth example, GAS did not recognize the triple "(bar menu,Disappointingly, negative)", while the method in this chapter successfully captured this information. In this sequence, "Disappointingly" and its corresponding attribute word "bar menu" are far apart, resulting in GAS failing to establish a connection between the two. The method in this chapter not only learns the semantic connection between them from the term pair matching module, but also captures the negative emotion contained in "Disappointingly" from the sentiment judgment module, thereby successfully identifying this triple.

[0079] In summary, by introducing multi-level pairing information enhancement, the extraction effect of attribute-level sentiment triples can be significantly improved.

[0080] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for extracting attribute-level sentiment triples based on pairing information enhancement, characterized in that: include, Get a review text dataset containing user reviews; Designing an attribute-level sentiment triple generation baseline model based on an encoder-decoder structure, wherein the attribute-level sentiment triple generation baseline model includes a T5 encoder and a T5 decoder; The review text is encoded into a hidden representation using a T5 encoder, and based on the hidden representation, an attribute-level sentiment triple is generated through a T5 decoder, wherein the attribute-level sentiment triple includes two output paradigms: annotative and extractive; The cross entropy loss function is used to calculate the triple generation loss ; Design the description information of term pairing knowledge and sentiment judgment knowledge, extract the dense vector representation of attribute-opinion pairs from the encoder output of the attribute-level sentiment triple generation baseline model, and perform multi-level pairing information enhancement; InfoNCE loss function is used to calculate the pairing layer contrast loss Compared with the sentiment layer loss ; Combining triplet generation loss , Paired Layer Contrastive Loss Compared with the sentiment layer loss , construct a comprehensive loss function , conduct multi-task joint training; Use the trained attribute-level sentiment triples to generate a baseline model and extract attribute-level sentiment triplets from new review texts.

2. The attribute-level sentiment triple extraction method based on pairing information enhancement as claimed in claim 1, characterized in that: The specific formula of the T5 encoder is as follows: ; in, is the output hidden representation of the T5 encoder, d is the dimension of the hidden representation, For T5 encoder, is the word sequence of the input comment text, n is the sequence length, and R is a set of real numbers; The specific formula of the T5 decoder is as follows: ; in, is the output of the T5 decoder at time step t, d is the dimension of the hidden representation, For T5 decoder, and are the learnable parameter matrices and bias vectors, for Activation function, is the output sequence of the decoder before time step t, is the vocabulary probability distribution at the current time step t, is the output hidden representation of the T5 encoder, R is a real number set, and n is the sequence length.

3. The attribute-level sentiment triple extraction method based on pairing information enhancement as claimed in claim 1, characterized in that: The multi-level pairing information enhancement includes pairing layer information enhancement and emotion layer information enhancement. The execution steps of the pairing layer information enhancement are as follows: Defining a description information set, wherein the description information set includes a matching type description and a non-matching type description; Encode the paired description information based on the BERT model to obtain the description information representation; Extract the hidden representations of attribute words and opinion words from the output of the T5 encoder, randomly concatenate the hidden representations of attribute words and opinion words, and obtain term pair representations through a linear layer; Using a contrastive learning method, calculating the similarity between the description information representation and the term pair representation; The paired layer contrast loss function is used to optimize the parameters of the baseline model for attribute-level sentiment triple generation and enhance the matching of attribute words and opinion words.

4. The attribute-level sentiment triple extraction method based on pairing information enhancement as claimed in claim 3 is characterized by: The specific formula for describing the information is as follows: ; in, for The output of the position, and is a learnable parameter, is the pairing information description, m represents the number of words, is the pre-trained bidirectional Transformer encoder function, is the activation function, The length of this sequence unit to describe the information; The specific formula for characterizing the term pair is as follows: ; in, and are the dense representations of attribute words and opinion words, respectively. is an average pooling function, and is a learnable parameter, Comment text The position index of attribute words and opinion words in , is the output hidden representation of the T5 encoder, is the term pair representation, are the starting position and ending position index of the attribute word respectively. are the starting and ending position indexes of the opinion word, respectively. This is a vector concatenation operation.

5. The attribute-level sentiment triple extraction method based on pairing information enhancement as claimed in claim 3, characterized in that: The execution steps of the emotional layer information enhancement are as follows: Defining a sentiment category set, wherein the sentiment category set includes three sentiment categories: positive, neutral and negative; Encode the emotional category description based on the BERT model and use a new linear layer to obtain emotional information representation; Extract the hidden representation of the real attribute word and opinion word pair from the output of the T5 encoder; The hidden representation of the true term pair is converted into a term pair representation through a linear layer; Using a contrastive learning method, calculating the similarity between the sentiment information representation and the term pair representation; Based on the contrast loss function of the emotion layer, the parameters of the attribute-level emotion triple generation baseline model are optimized to improve the ability of the attribute-level emotion triple generation baseline model to understand and distinguish different emotions. The emotional information representation The specific formula is as follows: ; in, is the activation function, for The output of the position, and are learnable parameters.

6. The attribute-level sentiment triple extraction method based on pairing information enhancement as claimed in claim 1, characterized in that: The sentiment layer contrast loss The specific formula is as follows: ; in, is the set of representations for all true term pairs that do not belong to the current sentiment category, To represent emotional information, is the real term pair information in the sequence, is the temperature coefficient, For real term information and the similarity of emotional information representation.

7. The attribute-level sentiment triple extraction method based on pairing information enhancement as claimed in claim 1, characterized in that: The comprehensive loss function The specific formula is as follows: ; in, is the weight hyperparameter, They are triplet generation loss, pairing layer contrast loss, and sentiment layer contrast loss respectively.

8. A system for extracting attribute-level sentiment triples based on pairing information enhancement, based on the method for extracting attribute-level sentiment triples based on pairing information enhancement according to any one of claims 1 to 7, characterized in that: Also includes, A text data acquisition module is used to obtain a review text dataset containing user reviews; Baseline model building module, used to build an attribute-level sentiment triple generation baseline model based on the encoder-decoder structure; A triple generation module is used to encode the comment text into a hidden representation using a T5 encoder, and based on the hidden representation, generate an attribute-level sentiment triple through a T5 decoder, wherein the attribute-level sentiment triple includes two output paradigms: annotative and extractive; Loss calculation module, used to calculate triple generation loss , Paired Layer Contrastive Loss Compared with the sentiment layer loss ; The multi-level pairing information enhancement module is used to design the description information of term pairing knowledge and sentiment judgment knowledge, extract the dense vector representation of attribute-opinion pairs from the encoder output of the triple generation model, and perform multi-level pairing information enhancement; Multi-task joint training module for combining triple generation losses , Paired Layer Contrastive Loss Compared with the sentiment layer loss , construct a comprehensive loss function , and conduct multi-task joint training; The attribute-level sentiment triplet generation module is used to use the trained attribute-level sentiment triplet generation baseline model to extract attribute-level sentiment triplets from new review texts.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the attribute-level sentiment triple extraction method based on pairing information enhancement as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the attribute-level sentiment triple extraction method based on pairing information enhancement according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Aspect-level emotion triple extraction method and system based on semantic enhancement double encoders

    CN116663567A

  • Bidirectional emotion triple extraction method based on span level and adversarial training

    CN117033631A

  • Intelligent voice reading method and system based on sentiment analysis

    CN119047477A

Cited By

  • Emotion ternary extraction model optimization method, recognition method, equipment and storage medium

    CN120688570A