Retention mechanism-based aspect emotion tetrad extraction method and system
Through a retention mechanism-based method, random data augmentation, pre-trained language model and multi-layer network analysis, combined with end decoding algorithms, the error propagation and inefficient computing problems of the quad prediction model in the prior art are solved, and efficient and accurate aspect emotional quad extraction is achieved.
Patent Information
- Application Number
- CN202510519900.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-24
- Publication Date
- 2025-07-25
AI Technical Summary
In the prior art, in sentiment analysis, the quadruple prediction model is prone to error propagation or inefficient calculations, especially in complex sentences, the extraction of emotional quadruple faces a large amount of information and complex word pair interactions, making it difficult to efficiently extract emotional quadruple.
Using a retention mechanism-based approach, aspect-view pairs and their emotional tendencies and category terms are separated through random data augmentation, pre-trained language models, fine-grained and coarse-grained resnet layer analysis, bidirectional residing layer processing, and end-decoding algorithms.
It improves the effectiveness of quadruple extraction, can effectively deal with remote complex quadruple extraction in complex sentences, and improves the accuracy and efficiency of sentiment analysis.
Smart Images
Figure CN120373313A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of natural language processing, and particularly relates to a method and system for extracting aspect sentiment quadruples based on a retention mechanism. Background Art
[0002] As an important branch task of natural language processing, sentiment analysis aims to effectively analyze and identify the sentiment tendencies contained in a large amount of text. With the rise of a large amount of text data in the Internet era, the huge emotional tendency value contained therein urgently needs to be applied to actual economic life, and sentiment text analysis has become increasingly important. Aspect sentiment quadruple prediction (ASQP) is a kind of sentiment analysis method in the prior art and has received extensive attention. It needs to extract four elements, namely aspect, category, opinion, and sentiment (a, c, o, s), from user comments. The extraction of the four-dimensional features can greatly expand the expression form of sentence semantics and bring more possibilities to the practical applications in the field of sentiment analysis.
[0003] Quadruple prediction models can be divided into pipeline methods and generation methods. The former solves sub-tasks by combining different techniques, such as using double propagation (DP) or JET to extract aspect-opinion-sentiment triples, and then detecting the corresponding category of the aspect and finally predicting the quadruple. Currently, most of the mainstream quadruple prediction models are generation methods; they model end-to-end prediction of quadruples through methods such as PARAPHRASE, and use the knowledge of pre-trained generation models to convert the target quadruple into a natural language sentence to solve the ASQP task. The mainstream methods include those based on tree structures, etc. However, existing methods may cause error propagation in pipeline-based strategies or inefficient calculations in generation-based methods. Moreover, the input compound sentence may contain multiple quadruples. As the complexity of the sentence increases, the amount of information to be transmitted during extraction is greater, and the word pair interaction elements and the overlapping tuple situations involved are more complex. Summary of the Invention
[0004] The purpose of the embodiments of the present invention is to provide a method for extracting aspect sentiment quadruples based on a retention mechanism, aiming to solve the problems proposed in the above background art.
[0005] The embodiments of the present invention are implemented as follows. The method for extracting aspect sentiment quadruples based on a retention mechanism includes the following steps:
[0006] Obtain the input text data;
[0007] Randomly perform four different data augmentation operations on the text data;
[0008] Preprocess the text data after data augmentation through a pre-trained language model, and output learnable word embedding vectors to represent the context relationship through the word embedding vectors;
[0009] Semantic parsing of word embedding vectors is performed on the word embedding vectors through a fine-grained ResNet layer and a coarse-grained ResNet layer respectively;
[0010] The result after semantic parsing is processed through a bidirectional retention layer;
[0011] Calculation is performed through an end decoding algorithm designed according to the characteristics of the quadruple, and the aspect-opinion pair and its corresponding sentiment tendency and category item are separated;
[0012] Output a set of quadruples.
[0013] Preferably, in the step of randomly performing four different data augmentation operations on the text data, the data augmentation operations respectively include:
[0014] Using synonym replacement to replace words other than the aspect item and the opinion item with synonyms;
[0015] Using back translation;
[0016] While maintaining the grammatical structure, perturb the word order to ensure that the aspect word and the opinion word maintain their relative positions;
[0017] Replace the opinion word with a word of similar sentiment.
[0018] Preferably, the step of preprocessing the text data after data augmentation through a pre-trained language model and outputting learnable word embedding vectors to represent the context relationship through the word embedding vectors is specifically:
[0019] H = [h1, h2,..., h n = BERT(x1, x2,..., x n ), H ∈ R |X|×dmodel
[0020] where (x1, x2, ···, x n ) are n vocabulary words of the input text data, BERT is the pre-trained language model, H is the learnable embedding vector output by BERT, and R |x|×dmodel represents the sentence length and dimension output within the real number range;
[0021] Then perform learning interaction:
[0022] w i = h i W, i ∈ [1, 102]
[0023] w j = h j U, j ∈ [1, 102]
[0024] M = w iT Vw j
[0025] Among them, h i and h j are the i-th and j-th word vectors in the sequence respectively; W and U are weight matrices, V is a weight matrix, T represents the transpose matrix, and M is the output learnable word embedding vector.
[0026] Preferably, the step of performing word vector semantic parsing on the word embedding vector through the fine-grained resnet layer and the coarse-grained resnet layer respectively is specifically:
[0027] M1 = sigmoid(ln(res al×al (M)))·M, M1 ∈ R n×n×r
[0028] M2 ∈ R n×n×r
[0029] R o ∈ R n×n×r
[0030] Among them, M1 and M2 are the output results of the fine-grained resnet layer and the coarse-grained resnet layer respectively, a1 and a2 both represent the number of loop layers, res() represents the resnet residual operation process, ln represents layer normalization, is the concatenation operation, sigmoid is the activation function, R represents the word pair relationship distribution within the real number range, n is the sentence length, r is the number of relationship types, and R o is the result of semantic parsing.
[0031] Preferably, the step of processing the result after semantic parsing through the bidirectional retention layer is specifically:
[0032] E = pool(R o )
[0033] E i = Biretention i (E), i ∈ [1, L]
[0034] Among them, pool is the pooling layer, L represents the number of layers of the bidirectional retention layer, and E i represents the output of the i-th layer of the bidirectional retention layer;
[0035] Perform the concatenation operation of group normalization, and finally enter the feedforward neural network to adjust the parameters:
[0036]
[0037] E o = FFN(ln(Y)) + Y, E o ∈R n×n×r
[0038] Among them, GroupNorm is the group normalization operation, and FFN is the feed-forward neural network.
[0039] Preferably, the step of calculating through the end decoding algorithm designed according to the characteristics of the quadruple and separating the aspect-opinion pair and its corresponding sentiment tendency and category item is specifically as follows:
[0040] E1[i, c] = E0[i, i, c]
[0041]
[0042] Among them, i, j ∈ (0, n), and c is the number of category items;
[0043] Connect the newly extended tensor with the output of the bidirectional retention layer:
[0044] C0 = Concat(E0, E2, E3, axis = 2)
[0045] Among them, Concat is the connection operation and is the second dimension;
[0046] Predict the category to which the aspect opinion item belongs by the softmax function:
[0047] P ij = softmax(linear(c ij ))
[0048] Among them, P ij represents the predicted category, c ij ∈C o , and linear is the linear function;
[0049] Input P ij into the decoding algorithm. The quantity represented by each third dimension represents the relationship information between word pairs, that is, the upper triangle is the same as the lower triangle. First, initialize the list of aspect items, opinion items, and sentiment polarities or category items to be separately extracted, and then traverse the rows and columns in turn to find the positions where multiple word pairs represent the same sentiment tendency or category item, and add the corresponding possible aspect items, opinion items, and sentiment polarities or category items to the corresponding A, O, S, and C lists. Finally, judge in turn whether the elements in each pair of A and O lists are aspect items and opinion items. If so, add them to the T list. If the opposite is true, swap and add them to the T list. Otherwise, do not add them. At the same time, the corresponding S and C are also added to the T list accordingly.
[0050] Another object of the embodiments of the present invention is to provide an aspect sentiment quadruple extraction system based on a retention mechanism for implementing the above-mentioned aspect sentiment quadruple extraction method based on a retention mechanism, including:
[0051] An acquisition module for acquiring the input text data;
[0052] A data enhancement module for randomly performing four different data enhancement operations on the text data;
[0053] A preprocessing module for preprocessing the text data after data enhancement through a pre-trained language model, outputting learnable word embedding vectors, and representing the context relationship through the word embedding vectors;
[0054] An analysis module for performing word vector semantic analysis on the word embedding vectors through a fine-grained ResNet layer and a coarse-grained ResNet layer respectively;
[0055] A bidirectional retention module for processing the results after semantic analysis through a bidirectional retention layer;
[0056] An end decoding module for calculating through an end decoding algorithm designed according to the characteristics of the quadruple, and separating the aspect-opinion pair and its corresponding sentiment tendency and category item;
[0057] An output module for outputting a quadruple set.
[0058] The aspect sentiment quadruple extraction method based on a retention mechanism provided by the embodiments of the present invention, on the basis of the extraction by the fine-grained and coarse-grained ResNet, combines the bidirectional retention strategy method, and adds the application of four random data enhancements. Finally, combined with the extraction characteristics of the aspect sentiment quadruple, a special end decoding module is designed, which improves the extraction efficiency of the quadruple and can also show good performance in dealing with remote complex quadruple extraction. Description of the Drawings
[0059] Figure 1 It is a flowchart of the aspect sentiment quadruple extraction method based on a retention mechanism provided by the embodiments of the present invention;
[0060] Figure 2 It is a schematic diagram of synonym replacement in data enhancement processing provided by the embodiments of the present invention;
[0061] Figure 3 It is a schematic diagram of back translation in data enhancement processing provided by the embodiments of the present invention;
[0062] Figure 4 It is a framework diagram of the aspect sentiment quadruple extraction system based on a retention mechanism provided by the embodiments of the present invention. Detailed Embodiments
[0063] In order to make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0064] The following describes in detail the specific implementation of the present invention with reference to specific embodiments.
[0065] As Figure 1 shown, the flowchart of the aspect sentiment quadruple extraction method based on the retention mechanism provided by an embodiment of the present invention includes the following steps:
[0066] S1. Obtain the input text data;
[0067] S2. Randomly perform four different data augmentation operations on the text data:
[0068] The data augmentation operations are divided into four different mechanisms, which are respectively:
[0069] Use synonym replacement to replace the words other than the aspect terms and opinion terms with synonyms. As Figure 2 shown, "food" is replaced with "meal", "amazing" is replaced with "fantastic", and "excellent" is replaced with "outstanding";
[0070] Use back translation for data augmentation, such as in the way of "English - French - English" or "English - German - English". As Figure 3 shown, the original sentence is first translated into French and then back into English, which may generate different expressions. In the original sentence, "hotel" is replaced with "inn", "comfortable" is replaced with "cozy", "staff" is replaced with "personnel", and "friendly" is replaced with "welcoming";
[0071] Disturb the word order while maintaining the grammatical structure to ensure that the aspect words and opinion words maintain their relative positions;
[0072] Replace the opinion words with words of similar sentiment, such as "positive": ["good", "great", "excellent", "nice", "wonderful", "fantastic"]; "negative": ["bad", "poor", "terrible", "awful", "horrible", "disappointing"]; "neutral": ["okay", "average", "moderate", "fair", "decent"];
[0073] However, if data augmentation operations are performed on all the training set data, it will lead to an overly large amount of data and poor final experimental results. Therefore, through experiments, the embodiments of the present invention randomly perform data augmentation on 40% of the training set, and for each text data in this 40% of the data, one of the above augmentation methods is randomly performed;
[0074] S3. Preprocess the text data after data augmentation through a pre-trained language model, output learnable word embedding vectors, represent the context relationship through the word embedding vectors, and perform two-dimensional modeling of the representation:
[0075] H = [h1, h2,..., h n = BERT(x1, x2,…, x n ), H ∈ R |X|×dmodel
[0076] Among them, (x1, x2, ···, x n ) are n vocabulary words of the input text data, BERT is the pre-trained language model, H is the learnable embedding vector output by BERT, and R |x|×dmodel represents the sentence length and dimension output within the real number range;
[0077] Then perform learning interaction:
[0078] w i = h i W, i ∈ [1, 102]
[0079] w j = h j U, j ∈ [1, 102]
[0080] M = w i T Vw j
[0081] Among them, h i and h jThey are the i-th and j-th word vectors in the sequence respectively; W and U are weight matrices, V is a weight matrix, T is a transpose matrix, and M is the learnable word embedding vector output;
[0082] S4. Perform word vector semantic parsing on the word embedding vector M through the fine-grained resnet layer and the coarse-grained resnet layer respectively. Both resnet layers are multi-layer network structures:
[0083] M1 = sigmoid(ln(res a1×a1 (M)))·M, M1 ∈ R n×n×r
[0084] M2 ∈ R n×n×r
[0085] R o ∈ R n×n×r
[0086] Among them, M1 and M2 are the output results of the fine-grained resnet layer and the coarse-grained resnet layer respectively. a1 and a2 both represent the number of loop layers, res() represents the resnet residual operation process, ln represents layer normalization, is the concatenation operation, sigmoid is the activation function, R represents the word pair relationship distribution within the real number range, n is the sentence length, r is the number of relationship types, and R o is the result of semantic parsing;
[0087] S5. Process the result after semantic parsing through the bidirectional retention layer:
[0088] E = pool(R o )
[0089] E i = Biretention i (E), i ∈ [1, L]
[0090] Among them, pool is the pooling layer, L represents the number of layers of the bidirectional retention layer, and E i represents the output of the i-th layer of the bidirectional retention layer;
[0091] Perform the concatenation operation of group normalization and finally enter the feed-forward neural network to adjust the parameters:
[0092]
[0093] E o = FFN(ln(Y)) + Y, E o ∈ R n×n×r
[0094] Among them, GroupNorm is the group normalization operation, and FFN is the feed-forward neural network;
[0095] S6. Calculate through the end decoding algorithm designed for the characteristics of the quadruple, and separate the aspect-opinion pair and its corresponding sentiment tendency and category item from the table representation:
[0096] E1[i, c] = E0[i, i, c]
[0097]
[0098] Among them, i, j ∈ (0, n), and c is the number of category items;
[0099] Connect the newly expanded tensor with the output of the bidirectional retention layer:
[0100] C0 = Concat(E0, E2, E3, axis = 2)
[0101] Among them, Concat is the connection operation and is the second dimension;
[0102] Predict the category to which the aspect opinion item belongs by the softmax function:
[0103] P ij = softmax(linear(c ij ))
[0104] Among them, P ij represents the predicted category, c ij ∈ C o , and linear is the linear function;
[0105] The corresponding decoding algorithm is as follows:
[0106]
[0107]
[0108] Connect P ijIn the input decoding algorithm, the quantity represented by each third dimension represents the relationship information between word pairs, that is, the upper triangle is the same as the lower triangle. First, initialize the list of aspect items, opinion items, and sentiment polarity or category items to be separately extracted, as shown in Step 1; then traverse the rows and columns in sequence to find the positions where the same sentiment tendency or category item is represented among multiple word pairs, and add the corresponding possible aspect items, opinion items, and sentiment polarity or category items to the corresponding A, O, S, and C lists, as shown in Steps 2-27; finally, judge in sequence whether the elements in each pair of A and O lists are aspect items and opinion items. If so, add them to the T list. If the opposite is true, swap and add them to the T list. Otherwise, do not add them. At the same time, the corresponding S and C are also added to the T list accordingly, as shown in Steps 28-34;
[0109] S7. Output the quadruple set.
[0110] As Figure 4 shown, it is a framework diagram of an aspect sentiment quadruple extraction system based on a retention mechanism provided by an embodiment of the present invention, including:
[0111] An acquisition module for acquiring the input text data;
[0112] A data enhancement module for randomly performing four different data enhancement operations on the text data;
[0113] A preprocessing module for preprocessing the text data after data enhancement through a pre-trained language model, outputting learnable word embedding vectors, and representing the context relationship through the word embedding vectors;
[0114] An analysis module for respectively performing word vector semantic analysis on the word embedding vectors through a fine-grained resnet layer and a coarse-grained resnet layer;
[0115] A bidirectional retention module for processing the results after semantic analysis through a bidirectional retention layer;
[0116] An end decoding module for calculating through an end decoding algorithm designed according to the characteristics of the quadruple, and separating the aspect-opinion pair and its corresponding sentiment tendency and category items;
[0117] An output module for outputting the quadruple set.
[0118] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. An aspect sentiment quadruple extraction method based on a retention mechanism, characterized in that It includes the following steps: Obtain the input text data; Randomly perform four different data augmentation operations on the text data; Preprocess the text data after data augmentation through a pre-trained language model, output learnable word embedding vectors, and represent the context relationship through the word embedding vectors; Semantically parse the word embedding vectors through a fine-grained ResNet layer and a coarse-grained ResNet layer respectively; Process the results after semantic parsing through a bidirectional retention layer; Perform calculations through an end decoding algorithm designed according to the characteristics of quadruples, and separate the aspect-opinion pairs and their corresponding sentiment tendencies and category items; Output a set of quadruples.
2. The method for extracting aspect sentiment quadruples based on the retention mechanism according to claim 1, wherein In the step of randomly performing four different data augmentation operations on the text data, the data augmentation operations respectively include: Use synonym replacement to replace the words other than the aspect item and the opinion item with synonyms; Use back translation; Disturb the word order while maintaining the grammatical structure to ensure that the aspect word and the opinion word maintain their relative positions; Replace the opinion word with a word of similar sentiment.
3. The method for extracting aspect sentiment quadruples based on the retention mechanism according to claim 1, characterized in that The step of preprocessing the text data after data augmentation through a pre-trained language model, outputting learnable word embedding vectors, and representing the context relationship through the word embedding vectors is specifically: H = [h1, h2, …, h n = BERT(x1, x2, …, x n ), H ∈ R|X|×dmodel Among them, (x1, x2, ···, x n ) are n words of the input text data, BERT is a pre-trained language model, and H is the learnable embedding vector output by BERT. R |x|×dmodel represents the sentence length and dimension output within the real number range; Perform learning interaction again: w i = h i W,i ∈ [1, 102] w j = h j U, j ∈ [1, 102] M = w i T Vw j where h i and h j are the i-th and j-th word vectors in the sequence respectively; W and U are weight matrices, V is a weight matrix, T denotes the transpose matrix, and M is the learnable word embedding vector output.
4. The method for extracting aspect sentiment quadruples based on the retention mechanism according to claim 3, wherein The step of semantically parsing the word embedding vectors through a fine-grained ResNet layer and a coarse-grained ResNet layer respectively is specifically: M1 = sigmoid(ln(resa1 × a1(M))) · M, M1 ∈ R n×n×f Among them, M1 and M2 are the output results of the fine-grained resnet layer and the coarse-grained resnet layer respectively, a1 and a2 both represent the number of loop layers, res() represents the resnet residual operation process, and ln represents layer normalization. is the concatenation operation, sigmoid is the activation function, R represents the distribution of word pair relationships within the real number range, n is the sentence length, r is the number of relationship types, and R o is the result of semantic parsing.
5. The aspect sentiment quadruple extraction method based on the retention mechanism according to claim 4, characterized in that The step of processing the results after semantic parsing through a bidirectional retention layer is specifically: E = pool(R o ) E i = Biretention i (E), i ∈ [1, L] Among them, pool is the pooling layer, L represents the number of layers of the bidirectional retention layer, and E i represents the output of the i-th layer of the bidirectional retention layer; Perform a concatenation operation of group normalization, and finally enter a feed-forward neural network to adjust the parameters: E o = FFN(ln(Y)) + Y, E o ∈R n×n×r Among them, GroupNorm is the group normalization operation, and FFN is the feed-forward neural network.
6. The method for extracting aspect sentiment quadruples based on the retention mechanism according to claim 5, characterized in that, The step of performing calculations through an end decoding algorithm designed according to the characteristics of quadruples, and separating the aspect-opinion pairs and their corresponding sentiment tendencies and category items is specifically: E1[i,c] = E0[i,i,c] Where i, j ∈ (0, n), and c is the number of category items; Concatenate the newly extended tensor with the output of the bidirectional retention layer: C0 = Concat(E0, E2, E3, axis = 2) Where Concat is the concatenation operation and is the second dimension; Predict the category to which the aspect opinion item belongs by the softmax function: P ij = softmax(linear(c ij )) where P ij represents the predicted class, c ij ∈ C o , and linear is a linear function; Input P ij In the input decoding algorithm, the quantity represented by each third dimension represents the relationship information between word pairs, that is, the upper triangle is the same as the lower triangle. First, initialize the list of aspect items, opinion items, and sentiment polarity or category items to be separately extracted. Then, traverse the rows and columns in turn to find the positions where the same sentiment tendency or category item is represented among multiple word pairs, and add the corresponding possible aspect items, opinion items, and sentiment polarity or category items to the corresponding A, O, S, and C lists. Finally, judge in turn whether the elements in each pair of A and O lists are aspect items and opinion items. If so, add them to the T list. If the opposite is true, swap and add them to the T list. Otherwise, do not add them. At the same time, the corresponding S and C are also added to the T list accordingly.
7. An aspect sentiment quadruple extraction system based on a retention mechanism, which is used to implement the aspect sentiment quadruple extraction method based on the retention mechanism as described in any one of claims 1-6, and is characterized in that It includes: An acquisition module for obtaining the input text data; A data augmentation module for randomly performing four different data augmentation operations on the text data; A preprocessing module for preprocessing the text data after data augmentation through a pre-trained language model, outputting learnable word embedding vectors, and representing the context relationship through the word embedding vectors; An analysis module for semantically parsing the word embedding vectors through a fine-grained ResNet layer and a coarse-grained ResNet layer respectively; A bidirectional retention module for processing the results after semantic parsing through a bidirectional retention layer; An end decoding module for performing calculations through an end decoding algorithm designed according to the characteristics of quadruples, and separating the aspect-opinion pairs and their corresponding sentiment tendencies and category items; An output module for outputting a set of quadruples.