A method and system for fine-grained emotion element extraction

Abstract

This invention proposes a fine-grained sentiment element extraction method and system, relating to the technical fields of deep learning and aspect sentiment analysis. The method involves inputting the acquired natural language text comment sequence into a pre-trained language model encoder for encoding, obtaining context-based representations. Sequence spans of different lengths are constructed, and the representations and span representations of different lengths are combined to form an original span representation sequence, which better captures contextual information. Implicit aspect word tags and opinion word tags containing local structural information are added before and after the original span representation sequence, laying the foundation for establishing the implicit connection between aspect words and opinion words. This sequence is then input into a pre-trained aspect sentiment quadruple extraction model for aspect sentiment quadruple extraction. This invention enables more fine-grained analysis of comment sequence information, accurately extracting the aspect sentiment quadruples corresponding to the natural language text comment sequence.

Description

Technical Field

[0001] This invention relates to the technical fields of deep learning and aspect-based sentiment analysis, and more specifically, to a method and system for fine-grained sentiment element extraction. Background Technology

[0002] With the widespread adoption of social media, online commentary, and digital communication, sentiment analysis has become a key tool for businesses to understand public sentiment, product reviews, and social trends. The task of sentiment analysis is to uncover the public's emotional tendencies, such as joy, anger, sorrow, and happiness. However, traditional sentiment analysis can no longer meet the current needs of businesses; more granular analysis is crucial for understanding user needs. For example, the sentence "I like this phone" contains positive emotion, but it's far from sufficient for practical application. Similarly, "The phone's camera is great, but its appearance is a bit ugly." Understanding this sentence alone makes it difficult to determine the emotional tendency towards the "phone" because "camera function" is positive, while "appearance" is negative. To analyze results more accurately, granular sentiment analysis methods are needed, and aspect-based sentiment analysis is one such widely discussed method. Aspect-based sentiment analysis can perform more comprehensive sentiment analysis, discovering various aspects of the text and determining the emotional information expressed by the text for each aspect. This makes sentiment analysis more precise and specific, contributing to a deeper understanding of the emotions people express in communication.

[0003] Aspect sentiment analysis focuses on extracting entity targets (also called aspect words a) from text and determining the sentiment information possessed by that aspect word a, such as opinion word o, aspect category c, and sentiment polarity s. A text may contain more than one of these aspect sentiment quadruples (aspect word a, opinion word o, aspect category c, and sentiment polarity s). If we classify text based on its representation, there are two main approaches. One is token-based representation, where the text is broken down into individual tokens (e.g., words or subwords), each treated as an independent unit and represented accordingly. This means each token has its own feature vector, and the model focuses on the features of individual tokens. In contrast to token representation, the other is span-based representation, which focuses on continuous segments or spans in the text, labeling or predicting spans with specific meanings or attributes. Span representation is easier to model for entire text segments and can better capture contextual information, especially effective for sequence labeling tasks. One existing technology discloses an aspect-sentiment triple extraction method. Based on three custom role dimensions—aspects, opinions, and sentiment fragments—it annotates each span in the text, then inputs the text sequence into a sentiment triple extraction model for aspect-sentiment triple extraction. This overcomes the limitations of existing word-level annotation methods and supports end-to-end extraction of corresponding aspects, opinions, and their corresponding matches, effectively improving the performance of aspect-sentiment triple extraction. In general, span representation considers the relationships between multiple tokens in the text, typically by aggregating all tokens within the span to obtain a holistic representation. Span representation, by aggregating information from multiple tokens in the text, can better capture the relationships within local text and is suitable for tasks requiring consideration of certain contextual information.

[0004] However, a significant challenge in fine-grained sentiment analysis is establishing connections between aspect terms and their corresponding opinion terms within comment statements, especially considering that a single comment may contain multiple aspect and opinion terms. This complexity stems from the diversity of comment statements, which vary in expression. Addressing this issue requires considering that different comment statements may have different forms of expression. Some comments may possess regular sentence structures, providing clues for distinguishing different pairs of aspect and opinion terms. Therefore, researching how to effectively utilize the regularity of sentence structure to more accurately establish connections between aspect and opinion terms for fine-grained sentiment element extraction has become a topic of profound research significance. Summary of the Invention

[0005] To address the challenge of accurately extracting sentiment quadruples and establishing connections between aspect terms and their corresponding viewpoint terms in fine-grained sentiment analysis, this application proposes a method and system for fine-grained sentiment element extraction, thereby improving the accuracy of fine-grained sentiment analysis.

[0006] To solve the above problems, the technical solution adopted in this application is as follows: On the one hand, this application proposes a fine-grained method for extracting sentiment elements, including the following steps: The acquired natural language text comment sequence is input into the pre-trained language model encoder for encoding, resulting in a context-based representation of the natural language text comment sequence; Construct the span of natural language text comment sequences of different lengths, combine the representations, construct span representations of different span lengths, form the original span representation sequence of all span representations, and add implicit aspect word tags and implicit opinion word tags containing local structural information before and after the original span representation sequence, respectively. The original span representation sequence, which includes implicit aspect word labels and implicit opinion word labels with added local structural information, is input into the pre-trained aspect sentiment quadruple extraction model to extract aspect sentiment quadruples.

[0007] Preferably, the aspect sentiment quadruple extraction model includes: For local structural modules, implicit aspect word prediction is performed based on implicit aspect word labels and the original span representation sequence; implicit opinion word prediction is performed based on implicit opinion word labels and the original span representation sequence. The aspect word and opinion word decoding module is used to classify the original span representation sequence to obtain the changed span representation sequence. Based on the original span representation sequence and the changed span representation sequence, the final representation of aspect words and the final representation of opinion words are predicted respectively. The aspect category prediction module is used to add the final representation of aspect words and the final representation of opinion words, and then concatenate them with the representation of the left span of aspect words and the representation of the right span of aspect words to obtain the category of aspect words. The interaction module for aspect words and opinion words obtains new representations of the final representations of aspect words and opinion words respectively. It combines the new representations of the final representations of aspect words with the final representations of aspect words and performs cross-attention mechanism calculations to obtain the opinion words and sentiment polarity probabilities corresponding to each aspect word. The reasoning, filtering, and merging module is used to merge the category of aspect words, the final representation of aspect words, the final representation of opinion words, and sentiment polarity, and to perform confidence filtering of aspect sentiment quadruples to extract the final aspect sentiment quadruples.

[0008] Preferably, the aspect sentiment quadruple extraction model is trained using the gradient descent method to obtain the trained aspect sentiment quadruple extraction model.

[0009] Preferably, the acquired natural language text comment sequence is represented as follows: Where N represents the length of the natural language text comment sequence, Indicates the first natural language text comment sequence i Each word, after being encoded by a pre-trained language model encoder, yields a context-based representation of the natural language text comment sequence, which is represented as follows: }; Set the maximum span length to L, and use a sliding window approach to construct the spans of natural language text comment sequences of different lengths from 1 to L:

[0010] ;

[0011] Combining the representations, span representations of different lengths are constructed. Multiple words within each length span constitute a span representation. From 1 to L, the length of the j-th span is represented as:

[0012] All span representations form the original span representation sequence, which is: Where start represents the starting position of the span in the context-based representation of the natural language text comment sequence, and end represents the ending position of the span in the context-based representation of the natural language text comment sequence; M This indicates the number of spans generated based on the maximum span length L;

[0013] Implicit aspect tags and implicit viewpoint tags containing local structural information are added before and after the original span representation sequence to generate the representation:

[0014] in, Indicates implicit aspect tag, The implicit opinion word label represents the local structural representation of the implicit aspect word and the implicit opinion word in the natural language text comment sequence output by the pre-trained language model encoder, respectively.

[0015] Based on the above technical means, after constructing span representations of different lengths, implicit aspect word tags and implicit opinion word tags are used to determine whether implicit aspect words or opinion words exist in the natural language text comment sequence.

[0016] Preferably, the local structure-oriented module includes a multi-head attention mechanism layer, a first multilayer perceptron, and a first softmax layer. Implicit aspect word tags are used as queries for the multi-head attention mechanism layer, the original span representation sequence is used as the key, and the original span representation sequence is used as the value. These are input to the multi-head attention mechanism layer for multi-head attention mechanism calculation. After the output of the multi-head attention mechanism layer, it is input to the first multilayer perceptron for implicit aspect word prediction. Finally, the prediction probability is obtained through the first softmax layer. The process satisfies the following expression: ;

[0017] The implicit opinion term tags are used as queries for the multi-head attention mechanism layer, the original span representation sequence is used as the key, and the original span representation sequence is used as the value. These are input to the multi-head attention mechanism layer for multi-head attention mechanism calculation. After the output of the multi-head attention mechanism layer, it is input to the first multilayer perceptron for implicit opinion term prediction. Finally, it passes through the first softmax layer to obtain the prediction probability. The process satisfies the following expression:

[0018] in, and These represent the probabilities of implicit aspect words and implicit opinion words existing in the original span representation sequence, respectively. This indicates the computation of the multi-head attention mechanism; This represents the prediction made by a multilayer perceptron.

[0019] Based on the above technical means, when multiple aspect sentiment quadruples appear in a natural language text comment sequence, the global context representation obtained by the implicit aspect word labels and implicit opinion word labels output by the pre-trained language model encoder contains mixed information. Extracting the representation information oriented towards the local structure can accurately predict whether there are implicit aspect words or opinion word groups in the sequence.

[0020] Preferably, the aspect and opinion word decoding module includes a dimensionality reduction layer, a restoration layer, a modified linear unit activation layer, a first splicing layer, a second multilayer perceptron, and a second softmax layer. In the aspect and opinion word decoding module, the process of classifying the original span representation sequence to obtain the changed span representation sequence includes: In the dimensionality reduction layer, the original span representation sequence is... Dimensionality reduction is performed, and then the original span representation sequence S after dimensionality reduction is restored to its original dimension in the reduction layer. Finally, it is input into the modified linear unit activation layer, and the output is the changed span representation sequence, which satisfies the expression:

[0021]

[0022] in, The span of change represents the sequence; This indicates the calculation of the modified linear unit activation layer. and Let represent the weight matrix and bias vector of the reduction operation in the reduction layer, respectively. and These represent the weight matrix and bias vector of the dimensionality reduction operation in the dimensionality reduction layer, respectively.

[0023] The process of predicting the final representations of aspect terms and viewpoint terms based on the original span representation sequence and the changed span representation sequence includes: In the first splicing layer, the original span representation sequence and the changed span representation sequence are spliced ​​together to obtain the spliced ​​span representation sequence. Satisfies the expression:

[0024] ;

[0025] In the second multilayer perceptron and the second softmax layer, based on the spliced ​​span representation sequence The predicted final representations of aspect terms and opinion terms are as follows:

[0026] =

[0027]

[0028] in, This represents the spliced ​​aspect term span representation sequence; This represents the spliced ​​sequence of opinion word spans; The final representation of aspect terms; The ultimate representation of opinion terms; This indicates the second-level perceptron computation for aspect word processing; This indicates the second-level perceptron computation for processing opinion terms.

[0029] Based on the above technical means, and based on span dimensionality reduction and restoration operations, each span of the original span representation sequence is classified to determine whether it is an aspect word or an opinion word. This is beneficial to retain important semantic information. Furthermore, different multilayer perceptrons enable the aspect sentiment quadruple extraction model to more accurately distinguish between aspect words and opinion words, reducing the impact of noise.

[0030] Preferably, the aspect category prediction module includes: a representation addition layer and a second splicing layer, wherein the final representation of the aspect words and the final representation of the viewpoint words are added together in the representation addition layer to obtain... + The representation of the aspect term is then concatenated with the representation of the left span of the aspect term and the representation of the right span of the aspect term to obtain the category of the aspect term, satisfying the expression:

[0031]

[0032] in, These represent the left-hand span of the aspect term and the right-hand span of the aspect term, respectively.

[0033] Based on the above technical means, considering the large number of categories in natural language text comment sequences, selection fusion and splicing are used to avoid the poor performance problem caused by simply using span representation. This can obtain richer contextual information and help to more accurately determine the category of aspect words.

[0034] Preferably, the aspect word and opinion word interaction module includes: an aspect word decoder, an opinion word decoder, a bidirectional long short-term memory (LSTM) layer, a cross-attention mechanism layer, an aspect word multilayer perceptron, an opinion word multilayer perceptron, and a third... softmax layer; The aspect term ultimately represents The final embedding representation of the aspect term decoder is passed through a bidirectional long short-time memory (LSTM) layer to obtain a new representation of the aspect term's final characterization. ;

[0035] The final representation of the viewpoints The final embedding representation of the opinion word decoder is passed through a bidirectional Long Short-Time Memory (LSTM) layer to obtain a new representation of the opinion word's final characterization. ;

[0036] The process of combining the new representation of the aspect term with the final representation of the aspect term, performing cross-attention mechanism calculation, and then obtaining the opinion term and sentiment polarity probability corresponding to each aspect term is as follows: Ultimately characterized by aspect terms As the keys and values ​​of the cross-attention mechanism layer, a new representation is ultimately represented by aspect words. As a query in the cross-attention mechanism layer, cross-attention mechanism calculation is performed, and then prediction is performed through an aspect-based multilayer perceptron to obtain the opinion word and sentiment polarity probability corresponding to each aspect word. The process satisfies:

[0037]

[0038]

[0039] in, This indicates the calculation of the cross-attention mechanism. This represents a novel span sequence representation after cross-attention mechanism computation on specific aspect words; Multilayer perceptron computation for aspect terms, This represents the probability of each span being an opinion word of a specific aspect, and represents the probability of sentiment polarity in four dimensions: non-aspect word-opinion word, positive aspect word-opinion word, neutral aspect word-opinion word, and negative aspect word-opinion word.

[0040] Ultimately represented by opinion words As the keys and values ​​of the cross-attention mechanism layer, a new representation of the final word representation. As a query in the cross-attention mechanism layer, cross-attention mechanism calculation is performed, and then prediction is performed through a multilayer perceptron for opinion words to obtain the aspect words and sentiment polarity probabilities corresponding to each opinion word. The process satisfies:

[0041]

[0042]

[0043] in, This indicates the calculation of the cross-attention mechanism. This represents a novel span sequence representation after applying a cross-attention mechanism to specific opinion terms. Multilayer perceptron computation represents opinion words. This represents the probability of each span being an aspect word of a specific opinion word, and represents the probability of the emotional polarity in four dimensions: non-aspect word-opinion word, positive aspect word-opinion word, neutral aspect word-opinion word, and negative aspect word-opinion word.

[0044] Based on the above technical means, the correspondence between aspect words and viewpoint words is learned. Through cross-attention mechanism calculation, interactive information representation is obtained. Through the cooperation of cross-attention mechanism layer, decoder, and bidirectional long short-term memory layer LSTM, the relationship between a certain aspect word and which spans are related is obtained, thereby obtaining the one-to-one or one-to-many relationship of aspect words.

[0045] Preferably, the reasoning filtering and merging module merges the category of aspect words, the final representation of aspect words, the final representation of opinion words, and the sentiment polarity, and performs confidence filtering of aspect sentiment quadruples to extract the final aspect sentiment quadruples as follows:

[0046] in, and These represent two sets of emotional quadruplets, representing emotional polarity. and They are different, but the categories of aspect terms, viewpoint terms, and aspect terms are the same; and They represent and emotional polarity The probability is used to compare the scores of the probability values ​​as the basis for determining the conflict resolution strategy when the emotional polarity of the aspect words and the viewpoint words are different, that is, when the emotional polarity is conflicted, and the final aspect emotional quadruple is determined.

[0047] The probability score is calculated by multiplying the probabilities of the aspect terms and their corresponding opinion terms, and the expression satisfies:

[0048]

[0049] in, Aspect-affective quadruples indicating the absence of a conflict in the emotional polarity between aspect words and opinion words. The final aspect-affective quadruple is extracted after resolving the conflict of affective polarity between aspect words and opinion words. This indicates the confidence level of the set aspect of the sentiment quadruple. and These represent the sets of aspect sentiment quadruples obtained after merging the two predictions; and These represent the probability of the aspect term and the probability of the corresponding opinion term, respectively. and These represent the probability of the opinion term and the probability of the corresponding aspect term, respectively.

[0050] Based on the aforementioned technical means, the category of aspect words, the final representation of aspect words, the final representation of opinion words, and the sentiment polarity are merged. During the two-way merging process, when the sentiment polarity of aspect words and opinion words conflicts, and the confidence of aspect sentiment quadruples causes the aspect sentiment quadruple extraction model to filter the output aspect sentiment quadruples, only retaining the quadruples that the model is highly confident in or have high scores, adopting the quadruples with higher sentiment polarity probabilities, and eliminating aspect sentiment quadruples with lower sentiment polarity probabilities.

[0051] On the other hand, the present invention proposes an aspect sentiment quadruplet extraction system based on span representation, the system comprising: a pre-trained language model encoder, a span representation generator, and a trained aspect sentiment quadruplet extraction model. The pre-trained language model encoder is used to encode the acquired natural language text comment sequence to obtain a context-based representation of the natural language text comment sequence; The span representation generator is used to construct the span of natural language text comment sequences of different lengths. Combined with the representation, span representations of different lengths are constructed. All span representations form the original span representation sequence. Implicit aspect word tags and implicit opinion word tags containing local structural information are added before and after the original span representation sequence, respectively. The trained aspect sentiment quadruple extraction model extracts aspect sentiment quadruples based on the original span representation sequence with added implicit aspect word labels and implicit opinion word labels.

[0052] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention proposes a fine-grained sentiment element extraction method and system. The method involves inputting the acquired natural language text comment sequence into a pre-trained language model encoder for encoding, obtaining a context-based representation of the natural language text comment sequence. Spans of natural language text comment sequences of different lengths are constructed, and combined with these representations, span representations of different lengths are built. All span representations form an original span representation sequence, better capturing contextual information. Implicit aspect word tags and implicit opinion word tags containing local structural information are added before and after the original span representation sequence, laying the foundation for establishing the connection between aspect words and opinion words. The original span representation sequence with these implicit aspect word tags and implicit opinion word tags is then input into a pre-trained aspect sentiment quadruple extraction model for aspect sentiment quadruple extraction. This invention enables more fine-grained analysis of natural language text comment sequence information, more accurately extracting the aspect quadruples corresponding to the natural language text comment sequence. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating the fine-grained emotional element extraction method proposed in this embodiment of the invention. Figure 2 This represents the global structure diagram for fine-grained emotional element extraction proposed in this embodiment of the invention; Figure 3 This is a schematic diagram illustrating the specific structure of the local structure module proposed in this embodiment of the invention. Figure 4 This is a schematic diagram illustrating the specific structure of the aspect term and viewpoint term decoding module proposed in this embodiment of the invention; Figure 5 This is a schematic diagram showing the specific structure of the aspect category prediction module proposed in this embodiment of the invention. Figure 6 This diagram illustrates the execution process within the aspect category prediction module proposed in this embodiment of the invention. Figure 7 This diagram illustrates the execution process within the aspect term and viewpoint term interaction module proposed in this embodiment of the invention. Figure 8 This is a schematic diagram illustrating the structure of the fine-grained emotional element extraction system proposed in this embodiment of the invention. Detailed Implementation

[0054] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this application. To better illustrate this embodiment, some parts of the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions; It is understandable to those skilled in the art that some well-known details may be omitted from the accompanying drawings.

[0055] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments; The positional relationships depicted in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this application. Example 1 This example proposes a fine-grained method for extracting emotional elements, such as... Figure 1 As shown, it includes the following steps: S1: Input the acquired natural language text comment sequence into the pre-trained language model encoder for encoding to obtain a context-based representation of the natural language text comment sequence; In this embodiment, the acquired natural language text comment sequence is represented as follows: Where N represents the length of the natural language text comment sequence, Indicates the first natural language text comment sequence i In this embodiment, the pre-trained language model encoder is a mature BERT encoder. After encoding by the pre-trained language model encoder, a context-based representation of the natural language text comment sequence is obtained, which is represented as follows: }

[0056] S2: Construct the span of natural language text comment sequences of different lengths, combine the representations to construct span representations of different lengths, all span representations form the original span representation sequence, and add implicit aspect word tags and implicit opinion word tags containing local structural information before and after the original span representation sequence respectively. In this embodiment, the maximum span length is set to L, and a sliding window approach is used to construct the spans of natural language text comment sequences of different lengths sequentially from 1 to L: ;

[0057] Combining the representations, span representations of different lengths are constructed. Multiple words within each length span constitute a span representation, ranging from 1 to L. In this embodiment, max pooling is used to aggregate multiple words within each span into a single overall representation. The length of the j-th span is represented as:

[0058] All span representations form the original span representation sequence, which is: Where start represents the starting position of the span in the context-based representation of the natural language text comment sequence, and end represents the ending position of the span in the context-based representation of the natural language text comment sequence; M This indicates the number of spans generated based on the maximum span length L;

[0059] Next, implicit aspect tags and implicit viewpoint tags containing local structural information are added before and after the original span representation sequence to generate the representation:

[0060] in, Indicates implicit aspect tag, The implicit opinion word label represents the local structural representation of the implicit aspect word and the implicit opinion word in the natural language text comment sequence output by the pre-trained language model encoder, respectively. They are used to determine whether there are implicit aspect words or opinion words in the natural language text comment sequence.

[0061] S3: Input the original span representation sequence containing implicit aspect word labels and implicit opinion word labels with local structural information into the trained aspect sentiment quadruple extraction model to perform aspect sentiment quadruple extraction.

[0062] Figure 2 For the global structure diagram of aspect sentiment quadruple extraction based on span representation proposed in this embodiment, see [link to diagram]. Figure 2 For example, the natural language sentence "It has great sushi and even better service." in S1 is encoded using a BERT encoder. Then, it constructs spans of natural language text comment sequences of different lengths, forming spans such as "[It], [has], [great], [sushi], [and], [even], [better], [service], [.], [It has][hasgreat]...". Combining these spans with representations, it constructs span representations of different lengths. All span representations form the original span representation sequence, and implicit aspect tags containing local structural information are added before and after the original span representation sequence. With implicit opinion tags .

[0063] In this embodiment, the aspect sentiment quadruple extraction model includes: a local structure-oriented module, an aspect word and opinion word decoding module, an aspect category prediction module, an aspect word and opinion word interaction module, and an inference, filtering, and merging module. Specifically, the local structure-oriented module performs implicit aspect word prediction based on implicit aspect word tags and the original span representation sequence; and performs implicit opinion word prediction based on implicit opinion word tags and the original span representation sequence. like Figure 3 As shown, the local structure module includes a multi-head attention mechanism layer, a first multilayer perceptron, and a first softmax layer. The implicit aspect term tags are used as queries for the multi-head attention mechanism layer, with the original span representation sequence as the key and the original span representation sequence as the value. These are input to the multi-head attention mechanism layer for multi-head attention mechanism calculation. The output of the multi-head attention mechanism layer is then input to the first multilayer perceptron for implicit aspect term prediction. Finally, the prediction probability is obtained through the first softmax layer. The process satisfies the following expression:

[0064] Similarly, implicit opinion word tags are used as queries for the multi-head attention mechanism layer, the original span representation sequence is used as the key, and the original span representation sequence is used as the value. These are input to the multi-head attention mechanism layer for multi-head attention mechanism calculation. After the output of the multi-head attention mechanism layer, it is input to the first multilayer perceptron for implicit opinion word prediction. Finally, it passes through the first softmax layer to obtain the prediction probability. The process satisfies the following expression:

[0065] in, and These represent the probabilities of implicit aspect words and implicit opinion words existing in the original span representation sequence, respectively. This indicates the computation of the multi-head attention mechanism; This represents the prediction made by a multilayer perceptron.

[0066] The aspect word and opinion word decoding module is used to classify the original span representation sequence to obtain the changed span representation sequence. Based on the original span representation sequence and the changed span representation sequence, the final representation of aspect words and the final representation of opinion words are predicted respectively. See Figure 4 The aspect and opinion word decoding module includes a dimensionality reduction layer, a restoration layer, a modified linear unit activation layer, a first splicing layer, a second multilayer perceptron, and a second softmax layer. In this module, the process of classifying the original span representation sequence to obtain the changed span representation sequence includes: In the dimensionality reduction layer, the original span representation sequence is... Dimensionality reduction is performed, and then the original span representation sequence S after dimensionality reduction is restored to its original dimension in the reduction layer. Finally, it is input into the modified linear unit activation layer, and the output is the changed span representation sequence, which satisfies the expression:

[0067]

[0068] in, The span of change represents the sequence; This indicates the calculation of the modified linear unit activation layer. and Let represent the weight matrix and bias vector of the reduction operation in the reduction layer, respectively. and These represent the weight matrix and bias vector of the dimensionality reduction operation in the dimensionality reduction layer, respectively.

[0069] The process of predicting the final representations of aspect terms and viewpoint terms based on the original span representation sequence and the changed span representation sequence includes: In the first splicing layer, the original span representation sequence and the changed span representation sequence are spliced ​​together to obtain the spliced ​​span representation sequence. Satisfies the expression:

[0070] ;

[0071] In the second multilayer perceptron and the second softmax layer, based on the spliced ​​span representation sequence The predicted final representations of aspect terms and opinion terms are as follows:

[0072] =

[0073]

[0074] in, This represents the spliced ​​aspect term span representation sequence; This represents the spliced ​​sequence of opinion word spans; The final representation of aspect terms; The ultimate representation of opinion terms; This indicates the second-level perceptron computation for aspect word processing; This indicates the second-level perceptron computation for opinion word processing. At this point... and There are two sets of parameters, one for aspect terms and one for opinion terms; therefore, the ones used... They are also different, which allows the model to more accurately distinguish between aspect terms and opinion terms, reducing the impact of noise.

[0075] The aspect category prediction module is used to add the final representation of aspect words and the final representation of opinion words, and then concatenate them with the representation of the left span of aspect words and the representation of the right span of aspect words to obtain the category of aspect words. like Figure 5 As shown, the aspect category prediction module includes a representation addition layer and a second splicing layer. Specifically, as shown, in the representation addition layer, the final representations of aspect words and opinion words are added together to obtain... + The representation of the aspect term is then concatenated with the representation of the left span of the aspect term and the representation of the right span of the aspect term to obtain the category of the aspect term, satisfying the expression:

[0076]

[0077] in, These represent the left-hand span of the aspect term and the right-hand span of the aspect term, respectively.

[0078] The interaction module for aspect words and opinion words obtains new representations of the final representations of aspect words and opinion words respectively. It combines the new representations of the final representations of aspect words with the final representations of aspect words and performs cross-attention mechanism calculations to obtain the opinion words and sentiment polarity probabilities corresponding to each aspect word. The aspect term and opinion term interaction module includes: aspect term decoder, opinion term decoder, bidirectional long short-term memory (LSTM) layer, cross-attention mechanism layer, aspect term multilayer perceptron, opinion term multilayer perceptron, and third... softmax layer; See Figure 7 The final representation of the aspect words The final embedding representation of the aspect term decoder is passed through a bidirectional long short-time memory (LSTM) layer to obtain a new representation of the aspect term's final characterization. Opinion words ultimately represent The final embedding representation of the opinion word decoder is passed through a bidirectional Long Short-Time Memory (LSTM) layer to obtain a new representation of the opinion word's final characterization. ;

[0079] The process of combining the new representation of the aspect term with the final representation of the aspect term, performing cross-attention mechanism calculation, and then obtaining the opinion term and sentiment polarity probability corresponding to each aspect term is as follows: Ultimately characterized by aspect terms As the keys and values ​​of the cross-attention mechanism layer, a new representation is ultimately represented by aspect words. As a query in the cross-attention mechanism layer, cross-attention mechanism calculation is performed, and then prediction is performed through an aspect-based multilayer perceptron to obtain the opinion word and sentiment polarity probability corresponding to each aspect word. The process satisfies:

[0080]

[0081]

[0082] in, This indicates the calculation of the cross-attention mechanism. This represents a novel span sequence representation after cross-attention mechanism computation on specific aspect words; Multilayer perceptron computation for aspect terms, This represents the probability of each span being an opinion word of a specific aspect, and represents the probability of sentiment polarity in four dimensions: non-aspect word-opinion word, positive aspect word-opinion word, neutral aspect word-opinion word, and negative aspect word-opinion word.

[0083] Ultimately represented by opinion words As the keys and values ​​of the cross-attention mechanism layer, a new representation of the final word representation. As a query in the cross-attention mechanism layer, cross-attention mechanism calculation is performed, and then prediction is performed through a multilayer perceptron for opinion words to obtain the aspect words and sentiment polarity probabilities corresponding to each opinion word. The process satisfies:

[0084]

[0085]

[0086] in, This indicates the calculation of the cross-attention mechanism. This represents a novel span sequence representation after applying a cross-attention mechanism to specific opinion terms. Multilayer perceptron computation represents opinion words. This represents the probability of each span being an aspect word of a specific opinion word, and represents the probability of the emotional polarity in four dimensions: non-aspect word-opinion word, positive aspect word-opinion word, neutral aspect word-opinion word, and negative aspect word-opinion word.

[0087] The reasoning, filtering, and merging module is used to merge the category of aspect words, the final representation of aspect words, the final representation of opinion words, and sentiment polarity, and to perform confidence filtering of aspect sentiment quadruples to extract the final aspect sentiment quadruples.

[0088] The reasoning filtering and merging module merges the category of aspect words, the final representation of aspect words, the final representation of opinion words, and sentiment polarity, and performs confidence filtering on aspect sentiment quadruples to extract the final aspect sentiment quadruples.

[0089] in, and These represent two sets of emotional quadruplets, representing emotional polarity. and They are different, but the categories of aspect terms, viewpoint terms, and aspect terms are the same; and They represent and emotional polarity The probability is used to compare the scores of the probability values ​​as the basis for determining the conflict resolution strategy when the emotional polarity of the aspect words and the viewpoint words are different, that is, when the emotional polarity is conflicted, and the final aspect emotional quadruple is determined.

[0090] The probability score is calculated by multiplying the probabilities of the aspect terms and their corresponding opinion terms, and the expression satisfies:

[0091]

[0092] in, Aspect-affective quadruples indicating the absence of a conflict in the emotional polarity between aspect words and opinion words. The final aspect-affective quadruple is extracted after resolving the conflict of affective polarity between aspect words and opinion words. This indicates the confidence level of the set aspect of the sentiment quadruple. and These represent the sets of aspect sentiment quadruples obtained after merging the two predictions; and These represent the probability of the aspect term and the probability of the corresponding opinion term, respectively. and These represent the probability of the opinion term and the probability of the corresponding aspect term, respectively.

[0093] Example 2 In this embodiment, for the aspect sentiment quadruple extraction model, the parameters of each layer in the model are first analyzed. Random initialization is performed, and then a loss function is constructed based on the true labels and predicted probabilities. This function includes category, aspect term, opinion term, and interaction loss between aspect and opinion terms. The parameters are then updated using the gradient descent algorithm. This yields a well-trained aspect-of-emotion quadruple extraction model.

[0094] Example 3 In this embodiment, a fine-grained emotional element extraction system is proposed, such as... Figure 8 As shown, the system includes: a pre-trained language model encoder, a span representation generator, and a pre-trained aspect sentiment quadruple extraction model. The pre-trained language model encoder is used to encode the acquired natural language text comment sequence to obtain a context-based representation of the natural language text comment sequence; The span representation generator is used to construct the span of natural language text comment sequences of different lengths. Combined with the representation, span representations of different lengths are constructed. All span representations form the original span representation sequence. Implicit aspect word tags and implicit opinion word tags containing local structural information are added before and after the original span representation sequence, respectively. The trained aspect sentiment quadruple extraction model extracts aspect sentiment quadruples based on the original span representation sequence with added aspect term labels and opinion term labels.

[0095] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for fine-grained extraction of emotional elements, characterized in that, Includes the following steps: The acquired natural language text comment sequence is input into the pre-trained language model encoder for encoding, resulting in a context-based representation of the natural language text comment sequence; Construct the span of natural language text comment sequences of different lengths, combine the representations, construct span representations of different span lengths, form the original span representation sequence of all span representations, and add implicit aspect word tags and implicit opinion word tags containing local structural information before and after the original span representation sequence, respectively. The original span representation sequences of implicit aspect word tags and implicit opinion word tags containing local structural information are input into the pre-trained aspect sentiment quadruple extraction model to perform aspect sentiment quadruple extraction; The aforementioned sentiment quadruple extraction model includes: For local structural modules, implicit aspect word prediction is performed based on implicit aspect word labels and the original span representation sequence; implicit opinion word prediction is performed based on implicit opinion word labels and the original span representation sequence. The aspect word and opinion word decoding module is used to classify the original span representation sequence to obtain the changed span representation sequence. Based on the original span representation sequence and the changed span representation sequence, the final representation of aspect words and the final representation of opinion words are predicted respectively. The aspect category prediction module is used to add the final representation of aspect words and the final representation of opinion words, and then concatenate them with the representation of the left span of aspect words and the representation of the right span of aspect words to obtain the category of aspect words. The interaction module for aspect words and opinion words obtains new representations of the final representations of aspect words and opinion words respectively. It combines the new representations of the final representations of aspect words with the final representations of aspect words and performs cross-attention mechanism calculations to obtain the opinion words and sentiment polarity probabilities corresponding to each aspect word. The reasoning, filtering, and merging module is used to merge the category of aspect words, the final representation of aspect words, the final representation of opinion words, and sentiment polarity, and to perform confidence filtering of aspect sentiment quadruples to extract the final aspect sentiment quadruples.

2. The fine-grained emotional element extraction method according to claim 1, characterized in that, The aspect sentiment quadruple extraction model was trained using the gradient descent method, resulting in a pre-trained aspect sentiment quadruple extraction model.

3. The fine-grained emotional element extraction method according to claim 1, characterized in that, The obtained natural language text comment sequence is represented as ,in, N Indicates the length of the natural language text comment sequence. Indicates the first natural language text comment sequence i Each word, after being encoded by a pre-trained language model encoder, yields a context-based representation of the natural language text comment sequence, which is represented as follows: }; Set the maximum span length to L, and use a sliding window approach to construct the spans of natural language text comment sequences of different lengths from 1 to L: ； Combining the representations, span representations of different lengths are constructed. Multiple words within each length span constitute a span representation. From 1 to L, the length of the j-th span is represented as: All span representations form the original span representation sequence, which is: Where start represents the starting position of the span in the context-based representation of the natural language text comment sequence, and end represents the ending position of the span in the context-based representation of the natural language text comment sequence; M This indicates the number of spans generated based on the maximum span length L; Implicit aspect tags and implicit viewpoint tags containing local structural information are added before and after the original span representation sequence to generate the representation: in, Indicates implicit aspect tag, The implicit opinion word label represents the local structural representation of the implicit aspect word and the implicit opinion word in the natural language text comment sequence output by the pre-trained language model encoder, respectively.

4. The fine-grained emotional element extraction method according to claim 3, characterized in that, The local structure-oriented module includes a multi-head attention mechanism layer, a first multilayer perceptron, and a first softmax layer; The implicit aspect term tags are used as queries for the multi-head attention mechanism layer, the original span representation sequence is used as the key, and the original span representation sequence is used as the value. These are input to the multi-head attention mechanism layer for multi-head attention mechanism calculation. After the output of the multi-head attention mechanism layer, it is input to the first multilayer perceptron for implicit aspect term prediction. Finally, it passes through the first softmax layer to obtain the prediction probability. The process satisfies the following expression: ； The implicit opinion term tags are used as queries for the multi-head attention mechanism layer, the original span representation sequence is used as the key, and the original span representation sequence is used as the value. These are input to the multi-head attention mechanism layer for multi-head attention mechanism calculation. After the output of the multi-head attention mechanism layer, it is input to the first multilayer perceptron for implicit opinion term prediction. Finally, it passes through the first softmax layer to obtain the prediction probability. The process satisfies the following expression: in, and These represent the probabilities of implicit aspect words and implicit opinion words existing in the original span representation sequence, respectively. This indicates the computation of the multi-head attention mechanism; This represents the prediction made by a multilayer perceptron.

5. The fine-grained emotional element extraction method according to claim 3, characterized in that, The aspect and opinion word decoding module includes a dimensionality reduction layer, a restoration layer, a modified linear unit activation layer, a first concatenation layer, a second multilayer perceptron, and a second softmax layer. In the aspect and opinion word decoding module, the process of classifying the original span representation sequence to obtain the changed span representation sequence includes: In the dimensionality reduction layer, the original span representation sequence is... Dimensionality reduction is performed, and then the original span representation sequence S after dimensionality reduction is restored to its original dimension in the reduction layer. Finally, it is input into the modified linear unit activation layer, and the output is the changed span representation sequence, which satisfies the expression: in, The span of change represents the sequence; This indicates the calculation of the modified linear unit activation layer. and Let represent the weight matrix and bias vector of the reduction operation in the reduction layer, respectively. and These represent the weight matrix and bias vector of the dimensionality reduction operation in the dimensionality reduction layer, respectively. The process of predicting the final representations of aspect terms and viewpoint terms based on the original span representation sequence and the changed span representation sequence includes: In the first splicing layer, the original span representation sequence and the changed span representation sequence are spliced ​​together to obtain the spliced ​​span representation sequence. Satisfies the expression: ； In the second multilayer perceptron and the second softmax layer, based on the spliced ​​span representation sequence The predicted final representations of aspect terms and opinion terms are as follows: = in, This represents the spliced ​​aspect term span representation sequence; This represents the spliced ​​sequence of opinion terms span representations; The final representation of aspect terms; The ultimate representation of opinion terms; This indicates the second-level perceptron computation for aspect word processing; This indicates the second-level perceptron computation for processing opinion terms.

6. The fine-grained emotional element extraction method according to claim 5, characterized in that, The aspect category prediction module includes a representation addition layer and a second splicing layer. In the representation addition layer, the final representations of aspect terms and opinion terms are added together to obtain the final representation. + The representation of the aspect term is then concatenated with the representation of the left span of the aspect term and the representation of the right span of the aspect term to obtain the category of the aspect term, satisfying the expression: ； in, These represent the left-hand span of the aspect term and the right-hand span of the aspect term, respectively.

7. The fine-grained emotional element extraction method according to claim 5, characterized in that, The aspect term and opinion term interaction module includes: aspect term decoder, opinion term decoder, bidirectional long short-term memory (LSTM) layer, cross-attention mechanism layer, aspect term multilayer perceptron, opinion term multilayer perceptron, and third... softmax layer; The aspect term ultimately represents The final embedding representation of the aspect term decoder is passed through a bidirectional long short-time memory (LSTM) layer to obtain a new representation of the aspect term's final characterization. ; The final representation of the viewpoints The final embedding representation of the opinion word decoder is passed through a bidirectional Long Short-Time Memory (LSTM) layer to obtain a new representation of the opinion word's final characterization. ; The process of combining the new representation of the aspect term with the final representation of the aspect term, performing cross-attention mechanism calculation, and then obtaining the opinion term and sentiment polarity probability corresponding to each aspect term is as follows: Ultimately characterized by aspect terms As the keys and values ​​of the cross-attention mechanism layer, a new representation is ultimately represented by aspect words. As a query in the cross-attention mechanism layer, cross-attention mechanism calculation is performed, and then prediction is performed through an aspect-based multilayer perceptron to obtain the opinion word and sentiment polarity probability corresponding to each aspect word. The process satisfies: in, This indicates the calculation of the cross-attention mechanism. This represents a novel span sequence representation after cross-attention mechanism computation on specific aspect words; Multilayer perceptron computation for aspect terms, This represents the probability of each span being an opinion word of a specific aspect, and represents the probability of sentiment polarity in four dimensions: non-aspect word-opinion word, positive aspect word-opinion word, neutral aspect word-opinion word, and negative aspect word-opinion word. Ultimately represented by opinion words As the keys and values ​​of the cross-attention mechanism layer, a new representation is ultimately represented by the opinion words. As a query in the cross-attention mechanism layer, cross-attention mechanism calculation is performed, and then prediction is performed through a multilayer perceptron for opinion words to obtain the aspect words and sentiment polarity probabilities corresponding to each opinion word. The process satisfies: in, This indicates the calculation of the cross-attention mechanism. This represents a novel span sequence representation after applying a cross-attention mechanism to specific opinion terms. Multilayer perceptron computation represents opinion words. This represents the probability of each span being an aspect word of a specific opinion word, and represents the probability of the emotional polarity in four dimensions: non-aspect word-opinion word, positive aspect word-opinion word, neutral aspect word-opinion word, and negative aspect word-opinion word.

8. The fine-grained emotional element extraction method according to claim 7, characterized in that, The reasoning filtering and merging module merges the category of aspect words, the final representation of aspect words, the final representation of opinion words, and sentiment polarity, and performs confidence filtering on aspect sentiment quadruples to extract the final aspect sentiment quadruples. in, and These represent two sets of emotional quadruplets, representing emotional polarity. and They are different, but the categories of aspect terms, viewpoint terms, and aspect terms are the same; and They represent and emotional polarity The probability is used to compare the scores of the probability values ​​as the basis for determining the conflict resolution strategy when the emotional polarity of the aspect words and the viewpoint words are different, that is, when the emotional polarity is conflicted, and the final aspect emotional quadruple is determined. The probability score is calculated by multiplying the probabilities of the aspect terms and their corresponding opinion terms, and the expression satisfies: in, Aspect-affective quadruples indicating the absence of a conflict in the emotional polarity between aspect words and opinion words. The final aspect-affective quadruple is extracted after resolving the conflict of affective polarity between aspect words and opinion words. This indicates the confidence level of the set aspect of the sentiment quadruple. and These represent the sets of aspect sentiment quadruples obtained after merging the two predictions; and These represent the probability of the aspect term and the probability of the corresponding opinion term, respectively. and These represent the probability of the opinion term and the probability of the corresponding aspect term, respectively.

9. A fine-grained emotional element extraction system, characterized in that, The system is used to implement the fine-grained sentiment element extraction method according to any one of claims 1 to 8, and the system includes: a pre-trained language model encoder, a span representation generator, and a trained aspect sentiment quadruple extraction model. The pre-trained language model encoder is used to encode the acquired natural language text comment sequence to obtain a context-based representation of the natural language text comment sequence; The span representation generator is used to construct the span of natural language text comment sequences of different lengths. Combined with the representation, span representations of different lengths are constructed. All span representations form the original span representation sequence. Implicit aspect word tags and implicit opinion word tags containing local structural information are added before and after the original span representation sequence, respectively. The trained aspect sentiment quadruple extraction model extracts aspect sentiment quadruples based on the original span representation sequence of the added implicit aspect word tags and implicit opinion word tags.

Images