A Method for Generating Sentiment Triples Based on Multi-Category Table Filling

Abstract

This invention provides a method for generating sentiment triples based on multi-class table filling, comprising the following steps: parsing the original comment text using a joint annotation framework and unifying the labels for aspect words, comment opinions, and sentiment polarity; extracting semantic features of text information using a BERT pre-trained language model; learning the association category-enhanced vector representation of aspect words and comment opinions using a multi-class multi-head attention mechanism; information partitioning and filtering for aspect word recognition and comment opinion detection tasks; using a unified labeling space for sentiment triples to achieve cell score filling and symmetry and implicit constraints on the table structure; performing unified label search and structured decoding based on the characteristic that aspect words, comment opinions, and sentiment polarity are all rectangular boxes in the unified labeling space; and constructing multi-functional sentiment triples for aspect words in comment text. This invention improves the accuracy of aspect word recognition and comment opinion detection and eliminates the problem of overlapping sentiment triples.

Description

Technical Field

[0001] This invention relates to the field of natural language processing information extraction technology, specifically to a method for generating sentiment triples based on multi-category table filling. Background Technology

[0002] With the rapid development of internet platforms such as social networking and e-commerce, more and more users are sharing their opinions on these platforms. A large number of user comments contain opinions and sentiments, and fine-grained opinion mining and sentiment analysis of these comments can yield more valuable information, which is of great significance to consumers, businesses, and governments. For example, user comments on events on social networking platforms can reveal users' stances on those events, while comments on e-commerce platforms can reveal users' satisfaction levels with goods and services. Currently, identifying aspect terms and extracting their corresponding sentiment polarities from comment texts has become a research hotspot in Aspect Sentiment Triplet Extraction (ASTE).

[0003] The purpose of aspect term sentiment triple extraction is to extract aspect term-opinion term triples (Aspect Term, Opinion Term, Sentiment, AOS) from a user review text. An aspect term, also known as an opinion target, is an entity word or phrase in the review text that represents the characteristics of a product or service; an opinion term is a word or phrase that expresses the user's attitude or viewpoint; and a sentiment term is the user's emotional polarity (positive, negative, neutral) towards the opinion target. For example, in the sentence "Makeup removal is very clean, gentle, and non-irritating; my face feels smooth and comfortable after using it," "makeup removal" is the aspect term, "very clean" is the opinion term, and "positive" is the sentiment term. Therefore, the aspect term sentiment triple for this sentence is ("makeup removal", "very clean", "positive").

[0004] Currently, there are two main approaches to aspect-word sentiment triple extraction: pipelined and joint extraction. The pipelined approach first uses two relatively independent sequence labeling models to extract aspect words and comment opinions. Then, it pairs the extracted aspect words and comment opinions, uses a classification model to determine the validity of the generated word pairs, and finally uses the valid word pair information to determine the sentiment polarity of the aspect word, thus generating the sentiment triple. However, this method has limitations in terms of triple extraction accuracy. The pipelined extraction method leads to cumulative errors; that is, the correctness of the aspect word-comment opinion pair extraction in the previous step affects the judgment of the sentiment polarity of the aspect word in the next step. The joint extraction approach can effectively mitigate the impact of error accumulation. It uses a multi-task framework to jointly detect aspect words, comment opinions, and sentiment dependencies. However, the framework still uses two independent sequence labeling models to identify and extract aspect words and comment opinions, ignoring the information interaction between them and failing to guarantee the sentiment consistency between word pairs.

[0005] In summary, although existing sentiment triple extraction methods have yielded some research results, they still have the following shortcomings: 1. Pipeline extraction methods can lead to error propagation, affecting the accuracy of triple extraction; 2. Joint extraction methods ignore the information interaction between aspect word extraction and comment opinion extraction, resulting in inconsistent sentiment polarity between word pairs; 3. In the joint extraction process, aspect word identification and comment opinion extraction ignore the influence of the type of aspect word or comment opinion; 4. They cannot handle the problem of overlapping sentiment triples, leading to low recognition efficiency; 5. When extracting aspect word-comment opinion pairs and predicting the sentiment polarity of word pairs, the label spaces between the two remain separate, hindering information interaction between them; 6. There is a lack of systematic analysis of the overall evaluation results of user reviews, meaning that although there are large-scale user product reviews as a reference, they cannot intuitively and quickly assist users in making decisions. Summary of the Invention

[0006] To address the technical problems existing in the prior art, this invention provides a method for generating sentiment triples based on multi-category table filling. By extracting detailed triple information from the comment text, it provides data support for sentiment triples of aspect words in user-oriented product reviews, thereby assisting users in making quick and accurate decisions.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0008] A method for generating sentiment triples based on multi-category table filling includes the following steps:

[0009] S1. First, clean the comment text information data obtained by the web crawler; second, uniformly label the comment opinions, evaluation objects (i.e., aspect words) and sentiment types in the data to construct a unified sentiment triple labeling space; finally, divide the labeled data into training set, validation set and test set in a ratio of 8:1:1.

[0010] S2. Use the BERT pre-trained language model to encode the features of the comment text, thereby extracting the deep semantic information H of the text;

[0011] S3. Based on the unified labeling space of the sentiment triple, use a multi-class multi-head attention mechanism to learn the category-enhanced vector representation H associated with the category of the comment and the aspect words. A And the augmented vector representation H of the association categories with comment opinions. O ;

[0012] S4. Represent H using category-enhanced vectors. A H O Based on this, a partitioning filtering mechanism is used to bidirectionally associate the aspect word recognition task with the comment opinion detection task. First, an aspect gate similar to an LSTM neural network is implemented using a linear layer neural network. and viewpoints Then, a gating mechanism is used to segment each time step unit into an aspect word recognition task partition ρ. A Comment and opinion detection task partition ρ O and shared task partition ρ S Finally, a filtering mechanism is used to filter out information irrelevant to the task, resulting in partition filtering information H. p ;

[0013] S5. Calculate the probability distribution score vector between each word pair using a dual affine deep attention mechanism, and fill the probability distribution score vector into each word pair cell of the two-dimensional table of the sentiment triplet unified label space;

[0014] S6. Add a symmetry constraint L to the uniform labels in the uniform label space of the sentiment triplet. sym and implicit constraints L imp ;

[0015] S7. Using the sentiment triplet unified label space joint decoding framework, traverse and search the squares representing aspect words and comments in the two-dimensional table and the rectangles representing sentiment polarity. First, use the property that adjacent rows or columns of aspect words or evaluation objects in the two-dimensional table are labeled in the same way to determine the boundary of the information. Second, use the property that squares are symmetric about the diagonal to decode aspect words or comments. Finally, use the detected aspect words and comments to traverse and search the sentiment polarity of the rectangular box structure aligned between aspect words and comments.

[0016] S8. Construct sentiment trigrams for aspect words in the comment text, aggregate the merits and demerits of sentiment evaluations for aspect words under each category and the reasons for their generation, summarize the sentiment trigrams of the overall comment text to reflect the overall evaluation results, and automatically generate feedback information based on the user's query conditions.

[0017] Furthermore, the construction of the unified labeling space for sentiment triples in step S1 includes the following steps:

[0018] S11. Obtain the starting and ending positions of the aspect words A and the opinion words O in the comment text, as well as the sentiment polarity Y of the corresponding aspect words. sent ={Pos,Neg,Neu};

[0019] S12. Obtain the category information between words describing various aspects and commentary viewpoints in the comment text, and statistically analyze to obtain m categories, defined as Y. c ={y1,y2,…,y m};

[0020] S13. Based on the obtained m category information, label aspect words, comment opinion tags, and sentiment polarity, and define the labeling method for aspect words as Y. A ={y1,…,y i ,None},y i ∈Y c Comments and opinions are marked with Y. O ={y1,…,y i ,None},y i ∈Y c The joint labeling method for emotional polarity is Y. P ={y1+p1,…,y i +p i ,None},y i ∈Y c p i ∈Y sent None indicates that there is no connection between word pairs;

[0021] S14. Fill the obtained aspect markers, comment / opinion markers, and sentiment polarity joint markers into Table T. n×n In each cell, to represent word pairs w i,j By identifying the information category relationships between them, a unified labeling space for sentiment triples is constructed, where n represents the length of the comment text S.

[0022] Furthermore, in step S3, a multi-class multi-head attention mechanism is used to learn the category-enhanced vector representation H associated with the category to which the comment belongs and the aspect words. A And the augmented vector representation H of the association categories with comment opinions.O Specifically, the following steps are included:

[0023] S31. Utilize the LSTM neural network model to further obtain deep semantic information of the text context at each time step. The detailed calculation method is as follows:

[0024]

[0025]

[0026]

[0027] Where W and b are trainable parameters, σ represents the sigmoid activation function, and i t o t f t These represent the input gate, output gate, and forget gate, respectively, and c t Indicates the cell state at the current time step; c t-1 Indicates the cell state at the previous time step; This indicates the cell state update value;

[0028] S32. Represent the Bert output vector. The hidden layer vector h output from the previous time step t-1 As the input to the multi-class multi-head attention mechanism module, firstly... and K (t) Dot product yields the semantic similarity 'a' between various categories and aspects of words or comments / opinions. (t) Then V (t) With a (t) Dot product yields an augmented vector representation of the aspect category or comment / opinion category. Finally, the hidden layer output vector of the LSTM neural network model is concatenated with the class augmentation vector representation to form the final vector representation h of this unit at time step. t The specific formula is as follows:

[0029]

[0030]

[0031]

[0032] Where softmax represents the activation function, d e The dimension of the word vectors output by BERT is represented by , and attention indicates the method of calculating the attention mechanism. m indicates the category to which the descriptive words or commentary belong. The key-value pair associated with the i-th category is shown in the following formula:

[0033]

[0034] Where σ represents the sigmoid activation function;

[0035] S33. Obtain the category-enhanced vector representations of the entire text sequence regarding aspect words and comment opinions, respectively. in The aspect terms are then concatenated with the category-enhanced vectors of the comments to obtain the final overall category-enhanced vector representation.

[0036] Furthermore, step S4, which utilizes a partitioning filtering mechanism to bidirectionally associate the aspect word recognition task with the comment opinion detection task, specifically includes the following steps:

[0037] S41, Utilizing the side door and viewpoints The information distribution for the aspect word recognition task and the comment opinion detection task are controlled separately. The neural unit at each time step is divided into an aspect word recognition task partition ρ. A Comment and opinion detection task partition ρ O and shared task partition ρ S The aspect gate and viewpoints The calculation method is shown in the following formula:

[0038]

[0039]

[0040] Where cummax represents the method for calculating the cumulative maximum value, and Linear represents the standard linear transformation. Let h represent the t-th overall category augmentation vector. t-1 This represents the hidden layer vector representation of the previous time step;

[0041] S42. Further calculate the current time step information representation.

[0042]

[0043] Where tanh represents the activation function;

[0044] S43. Calculate the aspect word recognition task partition corresponding to the historical time step. Comment and opinion detection task partition and shared task partition The vector information is represented and calculated as follows:

[0045]

[0046]

[0047]

[0048] in, This represents the element-wise multiplication operator. The calculation method and steps in S41 are as follows: and viewpoints The calculation method is consistent;

[0049] S44. Calculate the aspect word recognition task partition corresponding to the current time step. Comment and opinion detection task partition and shared task partition The vector information is represented and calculated as follows:

[0050]

[0051]

[0052]

[0053] S45. Add the partition vector of the current time step and the partition vector of the previous time step to form the overall partition information representation ρ of the current time step. A ρ O and ρ S The specific calculation method is shown in the following formula:

[0054]

[0055]

[0056]

[0057] S46. Utilize the pre-defined aspect terms to identify task partitions ρ A Comment and opinion detection task partition ρ O and shared task partition ρ S Different overlay methods are used to achieve information retention and filtering. Therefore, the memory storage unit for the aspect word recognition task partition, the comment opinion detection task partition, and the shared task partition is defined as μ, respectively. A μ O and μ S The specific calculation method is shown in the following formula:

[0058] μA =ρ A +ρ s μ O =ρ O +ρ s μ S =ρ s

[0059] S47. Concatenate the three types of partitioned memory storage units to obtain the cell state vector representation c for the next time step. t and hidden layer vector representation h t The specific calculation method is shown in the following formula:

[0060] c t =Linear([μ A,t μ O,t μ s,t ])

[0061] h t =tanh(c t )

[0062] Wherein, Linear represents a standard linear transformation;

[0063] S48. Finally, h is represented by a vector at each time step. t Based on this, we concatenate the data to generate H, a partitioning filtering information for aspect word recognition and comment opinion detection. p =[h1,h2,…,h n ].

[0064] Furthermore, step S5, which utilizes a dual affine deep attention mechanism to calculate the probability distribution score vector between each word pair and fills the probability distribution score vector into each word pair cell of the sentiment triplet unified label space two-dimensional table, specifically includes the following steps:

[0065] S51. Two multi-layer MLP neural network models are used to predict the beginning and end of each word. The specific calculation method is shown in the following formula:

[0066]

[0067] S52. Using a dual affine deep attention mechanism model, calculate the score vector representation g of each word pair according to the following formula. i,j :

[0068]

[0069]

[0070] Where Biaff represents the biaffine transformation, U1 and U2 are model weights, and b represents the offset;

[0071] S53. First, represent the fractional vector g. i,j The probability distribution of labels in word pairs is predicted by using the following formula as input to the softmax function:

[0072] P(y i,j |s)=softmax(dropout(g i,j ))

[0073] Then, the probability distribution of each word pair is filled into an n×n two-dimensional table T;

[0074] Finally, using the probability distribution of the predicted labels and the true labels, the overall loss value is calculated using the following formula:

[0075]

[0076] Among them, Y i,j This is a real label.

[0077] Furthermore, in step S6, a symmetry constraint L is added to the unified label in the unified label space two-dimensional table of the sentiment triplet. sym and implicit constraints L imp The steps are as follows:

[0078] S61, the annotation structure of aspect terms and commentary opinion terms are all squares symmetric about the diagonal. Therefore, the loss function with symmetry constraint is defined as L. sym :

[0079]

[0080] in, p(y) represents the p(y) of each word in the sentence for all possible labels. i,j |s) stacking;

[0081] S62. In the sentiment triplet, the sentiment polarity of the aspect words is closely related to both the aspect words and the commentary opinions. Therefore, the latent constraint loss function is defined as L. imp :

[0082]

[0083] in, p(y) represents the p(y) of each word in the sentence for all possible labels. i,j |s) stacking.

[0084] Compared with existing technologies, the sentiment triple generation method based on multi-category table filling provided by this invention has the following beneficial effects:

[0085] 1. The multi-class multi-head attention mechanism model described in this invention can take into account the influence of the category to which the comment text belongs on the aspect word recognition and comment opinion detection triples, which helps to improve the accuracy of aspect word recognition and comment opinion detection;

[0086] 2. The partitioned filtering neural network model described in this invention makes the two subtasks of aspect word recognition and comment opinion detection no longer isolated, but integrates the information between the two and divides them into three major partitions based on this: aspect word recognition task partition, comment opinion detection task partition, and shared task partition, thereby improving the bidirectional interaction between subtasks, preserving the information shared between the two, and discarding irrelevant information.

[0087] 3. The table-filling-based joint extraction framework described in this invention constructs a unified labeling space for sentiment triples, including comment opinions, aspect words, and sentiment polarity. It transforms the sequence labeling decoding method of aspect words and comment opinions into a method of searching rectangles in a two-dimensional table, thereby effectively eliminating information barriers between different subtasks and the problem of overlapping sentiment triples.

[0088] 4. The sentiment triplet of the comment text described in this invention enables rapid extraction of sentiment triplets from the comment text, so as to aggregate the advantages and disadvantages of sentiment of the words under each category and the reasons for their generation, and summarize the sentiment triplet of the overall comment text to reflect the overall evaluation results, and automatically generate feedback information according to the user's query conditions. Attached Figure Description

[0089] Figure 1 This is a schematic diagram of the emotional triple generation method based on multi-category table filling provided by the present invention.

[0090] Figure 2 This is a diagram of the overall model architecture for extracting emotion triples provided in this embodiment of the invention.

[0091] Figure 3 This is a detailed schematic diagram of the multi-class multi-head attention mechanism provided in the overall model architecture diagram of this invention.

[0092] Figure 4 This is a detailed schematic diagram of the partitioned filter machine in the overall model architecture diagram provided in the embodiments of the present invention.

[0093] Figure 5 This is an example diagram of a unified labeling space for emotion triples provided in an embodiment of the present invention. Detailed Implementation

[0094] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to specific illustrations.

[0095] Please refer to Figure 1 As shown, this invention provides a method for generating sentiment triples based on multi-category table filling, including the following steps:

[0096] S1. First, the comment text data obtained by the web crawler is cleaned. For example, the original user product comment text data crawled by the automated Selenium crawler is filtered and cleaned using data preprocessing methods such as regular expressions and manual review. This includes removing invalid characters and emoticons, and filtering out data whose comment text belongs to other categories. Second, the comment opinions, evaluation objects (i.e., aspect words), and sentiment types in the data are uniformly labeled to construct a unified sentiment triplet labeling space. Finally, the labeled data is divided into training, validation, and test sets in an 8:1:1 ratio. Subsequently, a sentiment triplet extraction model based on multi-class table filling is used to extract triplets. The overall structure of the model is shown in the attached figure. Figure 2 As shown;

[0097] S2. Use the BERT pre-trained language model to encode the features of the comment text, thereby extracting the deep semantic information H of the text;

[0098] S3. Based on the unified labeling space of the sentiment triple, use a multi-class multi-head attention mechanism to learn the category-enhanced vector representation H associated with the category of the comment and the aspect words. A And the augmented vector representation H of the association categories with comment opinions. O ;

[0099] S4. Represent H using category-enhanced vectors. A H O Based on this, a partitioning filtering mechanism is used to bidirectionally associate the aspect word recognition task with the comment opinion detection task. First, an aspect gate similar to an LSTM neural network is implemented using a linear layer neural network. and viewpoints Then, a gating mechanism is used to segment each time step unit into an aspect word recognition task partition ρ. A Comment and opinion detection task partition ρ O and shared task partition ρ S Finally, a filtering mechanism is used to filter out information irrelevant to the task, resulting in partition filtering information H. p This achieves two-way communication between tasks while avoiding the negative transfer problem of the task from other tasks.

[0100] S5. Calculate the probability distribution score vector between each word pair using a dual affine deep attention mechanism, and fill the probability distribution score vector into each word pair cell of the two-dimensional table of the sentiment triplet unified label space;

[0101] S6. Add a symmetry constraint L to the uniform labels in the uniform label space of the sentiment triplet. sym and implicit constraints L imp ;

[0102] S7. Using the sentiment triplet unified label space joint decoding framework, traverse and search the squares representing aspect words and comments in the two-dimensional table and the rectangles representing sentiment polarity. First, use the property that adjacent rows or columns of aspect words or evaluation objects in the two-dimensional table are labeled in the same way to determine the boundary of the information. Second, use the property that squares are symmetric about the diagonal to decode aspect words or comments. Finally, use the detected aspect words and comments to traverse and search the sentiment polarity of the rectangular box structure aligned between aspect words and comments.

[0103] S8. Construct sentiment triples for aspect words in the comment text to quickly extract sentiment triples from the comment text, aggregate the merits and demerits of sentiment evaluations for aspect words under each category and the reasons for their generation, summarize the sentiment triples of the overall comment text to reflect the overall evaluation results, and automatically generate feedback information based on the user's query conditions.

[0104] As a specific embodiment, the construction of the unified labeling space for emotion triples in step S1 includes the following steps:

[0105] S11. Analyze the preprocessed comment text information data to obtain the starting and ending positions of aspect words A and opinion words O in the comment text, as well as the sentiment polarity Y of the corresponding aspect words. sent ={Pos,Neg,Neu}, where the three sentiment polarity categories Pos, Neg, and Neu represent positive, negative, and neutral sentiments, respectively.

[0106] S12. Obtain the category information between words describing various aspects and commentary viewpoints in the comment text, and statistically analyze to obtain m categories, defined as Y. c ={y1,y2,…,y m For example, by analyzing a dataset of comment texts, 16 categories to which the comment texts belong are determined, and these categories are defined as Y. c ={y1,y2,…,y 16}, such as "logistics", "efficacy", "user experience", "price", "overall", "size", "smell", "authenticity", "packaging", "service", "ingredients", "freshness", "hardware performance", "usage scenario", "appearance", "software performance";

[0107] S13. Based on the obtained m category information, label aspect words, comment opinion tags, and sentiment polarity. That is, construct a unified labeling space using sentiment polarity labels and the category to which the comment text belongs, and define the labeling method of aspect words as Y.A ={y1,…,y i ,None},y i ∈Y c Comments and opinions are marked with Y. O ={y1,…,y i ,None},y i ∈Y c The joint labeling method for emotional polarity is Y. P ={y1+p1,…,y i +p i ,None},y i ∈Y c p i ∈Y sent None indicates that there is no relationship between the word pairs; for example, "Makeup removal is very clean, gentle and non-irritating, and my face feels smooth and comfortable after using it", where "makeup removal" is the aspect word, "very clean" is the comment opinion, the sentiment polarity is positive, and the category of the evaluation object is "efficacy", then the aspect word and the comment opinion are labeled as "efficacy" or "ECT", while the sentiment polarity category is jointly labeled as "ECT-POS", and the other words are labeled as "None";

[0108] S14. Fill the obtained aspect markers, comment / opinion markers, and sentiment polarity joint markers into Table T. n×n In each cell, to represent word pairs w i,j The information category relationships between them are used to construct a unified labeling space for sentiment triples, where n represents the length of the comment text S. The specific unified labeling space for sentiment triples is shown in the attached figure. Figure 5 As shown.

[0109] As a specific embodiment, step S2 utilizes a BERT pre-trained language model to perform feature encoding on the comment text, thereby extracting the deep semantic information H of the text. This specifically includes the following steps:

[0110] S21. Perform statistical analysis on the preprocessed comment text information data to obtain the longest sentence length in the comment text as 108. If the comment text length is less than 108 characters, the sequence needs to be padded with 0. Otherwise, if it exceeds 108 characters, it is truncated. At the same time, a [CLS] and a [SEP] character need to be concatenated and added at the beginning and end of the comment text respectively.

[0111] S22, such as Figure 2As shown, the Bert pre-trained language model is used for sentence encoding. The Bert pre-trained language model consists of the Encoder part of the 12-layer Transformer module. Each Encoder layer consists of Multi-Head Attention, LayerNormalization, and FeedForward. The Multi-Head Attention consists of 12 Heads. First, the word embedding representation of the comment text is multiplied by the three weight matrices to obtain the Query, Key, and Value matrices, as shown in Equation (1). Second, the semantic similarity α between the Query and Key is calculated, as shown in Equation (2). Then, the semantic similarity α is multiplied by the Value matrix to obtain the result of a single head, as shown in Equation (3). Finally, the results of the 12 Heads are merged to obtain the final output, as shown in Equation (4).

[0112] Query,Key,Value = X e (W Q W K W V Equation (1)

[0113]

[0114] head i =Attention(Query,Key,Value)=α i V-shape (3)

[0115] MultiHead(Query,Key,Value)=Concat(head1,…,head i W O Equation (4)

[0116] Furthermore, the output of the MultiHead is normalized using the LayNormalization module, and the final output of the current layer Encoder is obtained through analysis and processing by the feedforward neural network module. Each subsequent layer Encoder continues to repeat the above steps until the last layer Encoder outputs the deep semantic information H of the entire comment text, as shown in Equation 5 below:

[0117]

[0118] The output H of the Bert pre-trained language model in equation (5) is used as the data input for the multi-class multi-head attention mechanism. The multi-class multi-head attention mechanism model is applied to the aspect word recognition and comment opinion extraction modules respectively, so as to obtain the interaction information between comment categories, aspect words, and comment opinions, as shown in the appendix. Figure 2 The Aspect Type Attention and Opinion Type Attention sections are shown in the diagram. Each multi-class multi-head attention mechanism module consists of a Long Short-Term Memory (LSTM) neural network unit and a multi-class attention unit (Type-Attention). Therefore, in a specific embodiment, step S3 utilizes the multi-class multi-head attention mechanism to learn the category-enhanced vector representation H associated with the category to which the comment belongs and the aspect word, respectively. A And the augmented vector representation H of the association categories with comment opinions. O Specifically, the following steps are included:

[0119] S31. Utilize the LSTM neural network model to further obtain deep semantic information of the text context at each time step. The detailed calculation method is as follows:

[0120]

[0121]

[0122]

[0123] Where W and b are trainable parameters, σ represents the sigmoid activation function, and i t o t f t These represent the input gate, output gate, and forget gate, respectively, and c t Indicates the cell state at the current time step; c t-1 Indicates the cell state at the previous time step; This indicates the cell state update value;

[0124] S32. Represent the Bert output vector from step S31. The hidden layer vector h output from the previous time step t-1 As the input to the multi-category multi-head attention mechanism module, to highlight the relative influence of multiple comment categories on terms and comment opinions, the first step is to... and K (t) Dot product yields the semantic similarity 'a' between various categories and aspects of words or comments / opinions. (t) Then V (t) With a (t) Dot product yields an augmented vector representation of the aspect category or comment / opinion category. Finally, the hidden layer output vector of the LSTM neural network model is concatenated with the class augmentation vector representation to form the final vector representation h of this unit at time step. t The specific formula is as follows:

[0125]

[0126]

[0127]

[0128] Where softmax represents the activation function, d e The dimension of the word vectors output by BERT is represented by , and attention indicates the method of calculating the attention mechanism. m indicates the category to which the descriptive words or commentary belong. The key-value pair associated with the i-th category is shown in the following formula:

[0129]

[0130] Where σ represents the sigmoid activation function;

[0131] S33. Since aspect terms and comment opinions share the same comment text description category, the category-enhanced vector representations of the entire text sequence regarding aspect terms and comment opinions can be obtained through steps S31 and S32 above, respectively. in The aspect terms are then concatenated with the category-enhanced vectors of the comments to obtain the final overall category-enhanced vector representation. The detailed module structure is attached. Figure 3 As shown.

[0132] In step S33 above, the final overall category augmentation vector representation H is obtained. type Based on this, a task partitioning filtering neural network model, i.e., a partitioning filtering mechanism, is constructed, as shown in the attached figure. Figure 2 The Partition Filter section is shown below. This network model consists of a partition encoder and a partition filter encoder: the partition encoder uses a gating mechanism to divide each neuron into three regions: an aspect recognition task region, a comment / opinion detection task region, and a shared task region; the partition filter encoder is used to eliminate conflicting information between different tasks and prevent the propagation of error messages. Detailed module structure is shown in the attached figure. Figure 4 As shown. Therefore, in a specific embodiment, step S4, which uses a partitioning filtering mechanism to bidirectionally associate the aspect word recognition task with the comment opinion detection task, specifically includes the following steps:

[0133] S41, Utilizing the side door and viewpoints The information distribution for aspect recognition and opinion detection tasks is controlled separately. For any given task, the gating mechanism divides a neural unit into two parts: one containing information relevant to that specific task, and the other containing information unrelated to that task. Then, by combining the partitioning results from the two tasks, a shared task partition is formed. These two gating mechanisms are then defined as aspect gates. and viewpoints The aspect gate and viewpoints The calculation method is shown in the following formula:

[0134]

[0135]

[0136] Where cummax represents the method for calculating the cumulative maximum value, and Linear represents the standard linear transformation. h represents the t-th global category augmentation vector representation in step S33. t-1 This represents the hidden layer vector representation of the previous time step;

[0137] Using the aforementioned gated neural units, the neural units at each time step can be further divided into three partitions: the aspect word recognition task partition ρ. A Comment and opinion detection task partition ρ O and shared task partition ρ S ;

[0138] S42. To further calculate the information representation of the three partitions, it is necessary to rely on the information representation of the current time step before partitioning. c and the historical information from the previous time step t-1 The computation of candidate unit information representation borrows from the idea of ​​LSTM neural networks in calculating the memory storage unit of the current time step, and further computes the information representation of the current time step. The method is as follows:

[0139]

[0140] Where tanh represents the activation function;

[0141] S43. Utilize the historical time step information obtained in step S42. t-1 Calculate the aspect word recognition task partition corresponding to the historical time step. Comment and opinion detection task partition and shared task partition The vector information is represented and calculated as follows:

[0142]

[0143]

[0144]

[0145] in, This represents the element-wise multiplication operator. The calculation method and steps in S41 are as follows: and viewpoints The calculation method is consistent;

[0146] S44. Using the information of the current time step obtained in step S42 above, represent Calculate the aspect word recognition task partition corresponding to the current time step Comment and opinion detection task partition and shared task partition The vector information is represented and calculated as follows:

[0147]

[0148]

[0149]

[0150] in, This represents the element-wise multiplication operator;

[0151] S45. Add the partition vector of the current time step and the partition vector of the previous time step to form the overall partition information representation ρ of the current time step. A ρ O and ρ S The specific calculation method is shown in the following formula:

[0152]

[0153]

[0154]

[0155] in, This represents the element-wise multiplication operator;

[0156] S46. Further construct memory storage units for three types of partition information to ensure a high degree of consistency of relevant information and discard irrelevant information. To this end, utilize the already divided aspect word recognition task partitions ρ. A Comment and opinion detection task partition ρ O and shared task partition ρ SDifferent overlay methods are used to achieve information retention and filtering. The memory storage units for the aspect word recognition task partition, the comment opinion detection task partition, and the shared task partition are defined as μ, respectively. A μ O and μ S The specific calculation method is shown in the following formula:

[0157] μ A =ρ A +ρ s μ O =ρ O +ρ s μ S =ρ s Equation (25)

[0158] From the above equation, it can be seen that for μ A In this regard, its main information comes from the aspect word recognition task partition and the shared task partition; μ O The information comes from the comment opinion detection task partition and the shared task partition; μ S The information sources are mainly concentrated in the shared task partition;

[0159] S47. Concatenate the partition feature vector representation output by the partitioning encoder with the memory storage unit obtained by the partition filtering encoder, that is, concatenate the three partition memory storage units to obtain the unit state vector representation c for the next time step. t and hidden layer vector representation h t The specific calculation method is shown in the following formula:

[0160] c t =Linear([μ A,t μ O,t μ s,t Equation (26)

[0161] h t =tanh(c t Equation (27)

[0162] Wherein, Linear represents a standard linear transformation;

[0163] S48. Finally, h is represented by a vector at each time step. t Based on this, we concatenate the data to generate H, a partitioning filtering information for aspect word recognition and comment opinion detection. p =[h1,h2,…,h n ].

[0164] Further in H p Building upon this foundation, a joint extraction framework based on table filling is constructed: First, a dual affine deep attention mechanism is used to extract information from the previous layer H.p Map the words to their head and tail vectors in each word pair, and calculate the score or value g between each word pair. i,j and g i,j The scores are filled into a two-dimensional table T of the unified label space of the sentiment triplet; then, symmetry constraints and implicit constraints are added to the unified labels in the table through structural regularization; finally, the squares and rectangles in the two-dimensional table are identified by the unified decoding framework of sentiment triplet.

[0165] In a specific embodiment, step S5 utilizes a dual affine deep attention mechanism to calculate the probability distribution score vector between each word pair, and then fills the probability distribution score vector into each word pair cell of the sentiment triple unified label space two-dimensional table, as shown in the attached figure. Figure 2 The Biaffine Model section is shown below. Specifically, it includes the following steps:

[0166] S51. Two multi-layer MLP neural network models are used to predict the beginning and end of each word. The specific calculation method is shown in the following formula:

[0167]

[0168] S52. Using a dual affine deep attention mechanism model, calculate the score vector representation g of each word pair according to the following formula. i,j :

[0169]

[0170]

[0171] Where Biaff represents the biaffine transformation, U1 and U2 are model weights, and b represents the offset;

[0172] S53, after obtaining the fractional vector representation g i,j Next, the fractional vector g is first represented. i,j The softmax function is used as input to predict the labels in word pairs according to the following formula, thereby deriving the classification probability distribution on the label space γ:

[0173] P(y i,j |s)=softmax(dropout(g i,j Equation (31)

[0174] Then, the probability distribution of each word pair is filled into an n×n two-dimensional table T;

[0175] Finally, using the probability distribution of the predicted labels and the true labels, the overall loss value is calculated using the following formula:

[0176]

[0177] Among them, Y i,j This is a real label.

[0178] Appendix Figure 5 The unified label space for the sample sentiment triplet is symmetric about a two-dimensional table, and sentiment polarity necessarily depends on aspect words and comment opinions. Therefore, the symmetry and implicitness of the two-dimensional table are used to restrict aspect word recognition and comment opinion detection. Based on this, symmetry and implication constraints are added to the two-dimensional table T through structural regularization. Therefore, as a specific embodiment, step S6 adds a symmetry constraint L to the unified labels in the two-dimensional table of the sentiment triplet unified label space. sym and implicit constraints L imp The steps are as follows:

[0179] S61. The square corresponding to aspect terms in the two-dimensional table T must be symmetrical about the diagonal. Secondly, it is also symmetrical for comment / opinion tags. That is, the annotation structure of both aspect terms and comment / opinion tags are squares symmetrical about the diagonal. The sentiment triples (A1, O1, Sent) and (O1, A1, Sent) are equivalent. Therefore, symmetry constraints can be used to improve the recognition results. For this purpose, the loss function for symmetry constraints is defined as L. sym :

[0180]

[0181] in, p(y) represents the p(y) of each word in the sentence for all possible labels. i,j |s) stacking;

[0182] S62. In sentiment triples, the sentiment polarity of aspect words is necessarily closely related to both aspect words and comment opinions. If the sentiment polarity of aspect words exists, then both aspect words and comment opinions must exist. That is, the probability of the sentiment polarity of aspect words is never greater than the probability of both aspect words and comment opinions. Therefore, this implicit constraint can be easily added to the sentiment triple extraction task. For this purpose, the loss function for the implicit constraint is defined as L. imp :

[0183]

[0184] in, p(y) represents the p(y) of each word in the sentence for all possible labels. i,j |s) stacking.

[0185] Furthermore, the overall loss function L is minimized through joint training using the following formula. whole Symmetric constraint loss function L sym The implicit constraint loss function is L imp :

[0186] L = L whole +L sym +L imp

[0187] By observing the appendix Figure 5 The study found that both aspect terms and comment opinions are squares symmetrical about their diagonals, and the sentiment polarity between them is a rectangle about their diagonals. Therefore, the mining of sentiment triples is transformed into a search of rectangular boxes. Based on this, as a specific embodiment, step S7 utilizes a unified label space joint decoding framework for sentiment triples to detect squares and rectangles in the two-dimensional table T: First, the span range prediction of aspect terms and comment opinions is decoded, i.e., the boundary of the information is determined by using the property that adjacent rows or columns of aspect terms or evaluation objects in the two-dimensional table have consistent labels; then, the type of aspect terms and comment opinions is decoded, i.e., the aspect terms or comment opinions are decoded by using the property that squares are symmetrical about their diagonals; finally, using the detected aspect terms and comment opinions, the sentiment polarity of the rectangular box structures aligned between aspect terms and comment opinions is traversed and searched to decode the sentiment polarity between aspect terms and comment opinions, thereby generating sentiment triples. Specifically, it includes the following steps:

[0188] S71. Based on the property that the rows and columns corresponding to the same label in the two-dimensional table T of sentiment triples must have the same property, determine the boundaries of aspect words and commentary opinions:

[0189] First, expand the probability matrix p row by row to obtain... And calculate the Euclidean distance between adjacent rows;

[0190] Then expand the probability matrix p column by column to obtain And calculate the Euclidean distance between adjacent columns;

[0191] Finally, calculate the average distance between the two and compare it with the default distance threshold. If the distance exceeds this threshold, it indicates that the position is a split position.

[0192] S72. Using the property that aspect words and comments in the two-dimensional table T of sentiment triples are about a diagonally symmetric square, identify and detect the categories of the two. if Then it is decoded into terms or commentary opinions;

[0193] S73. Given that the span range of the aspect term a1 is (i,j) and the span range of the comment / opinion term o1 is (m,n), based on the correlation between the two in the two-dimensional table, decode the rectangular sentiment type between them. like This is then decoded into the emotional polarity between the two, thus forming the final emotional triad.

[0194] As a specific implementation, the above-mentioned sentiment triple model is constructed using the PyTorch deep learning framework, and trained on the training set and validated on the validation set. The training parameters of the model are saved, and the front-end and back-end deployment of the sentiment triple model is further implemented using the Flask framework, which includes the following steps:

[0195] First, the front-end page layout of the system is implemented using HTML.

[0196] Secondly, write backend programs to load, predict, and analyze the parameters of the sentiment triplet model;

[0197] Finally, the Flask framework is used to connect the page data and the backend interface to achieve a visualization effect.

[0198] Furthermore, the sentiment triplet for aspect words in the comment text constructed by this invention includes the following functions: identification and generation of sentiment triplets for comment text, aggregation of the merits and demerits of sentiment evaluations for aspect words under each category and the reasons for their generation, summarization of the overall sentiment triplet for the comment text to reflect the overall evaluation results, and automatic generation of feedback information that meets the user's query conditions.

[0199] Compared with existing technologies, the sentiment triple generation method based on multi-category table filling provided by this invention has the following beneficial effects:

[0200] 1. The multi-class multi-head attention mechanism model described in this invention can take into account the influence of the category to which the comment text belongs on the aspect word recognition and comment opinion detection triples, which helps to improve the accuracy of aspect word recognition and comment opinion detection;

[0201] 2. The partitioned filtering neural network model described in this invention makes the two subtasks of aspect word recognition and comment opinion detection no longer isolated, but integrates the information between the two and divides them into three major partitions based on this: aspect word recognition task partition, comment opinion detection task partition, and shared task partition, thereby improving the bidirectional interaction between subtasks, preserving the information shared between the two, and discarding irrelevant information.

[0202] 3. The table-filling-based joint extraction framework described in this invention constructs a unified labeling space for sentiment triples, including comment opinions, aspect words, and sentiment polarity. It transforms the sequence labeling decoding method of aspect words and comment opinions into a method of searching rectangles in a two-dimensional table, thereby effectively eliminating information barriers between different subtasks and the problem of overlapping sentiment triples.

[0203] 4. The sentiment triplet of the comment text described in this invention enables rapid extraction of sentiment triplets from the comment text, so as to aggregate the advantages and disadvantages of sentiment of the words under each category and the reasons for their generation, and summarize the sentiment triplet of the overall comment text to reflect the overall evaluation results, and automatically generate feedback information according to the user's query conditions.

[0204] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for generating sentiment triplets based on multi-class table filling, characterized in that, The method comprises the following steps: S1, first, the comment text information data obtained by the crawler tool is cleaned; second, the comment opinions, aspect words and sentiment types in the data are uniformly labeled to construct a sentiment triple unified labeling space; and finally, the labeled data is divided into a training set, a validation set and a test set in a ratio of 8:1:1; S2, using a Bert pre-training language model to encode features of the review text, thereby extracting deep semantic information of the text ; S3, according to the emotion triple unified marking space, using a multi-class multi-head attention mechanism to learn the category-enhanced vector representation associated with the aspect word of the category to which the review belongs respectively , and the associated category-enhanced vector representation of the review opinion ; S4, representing by category enhancement vector , Based on the above, the aspect term recognition task and the review opinion detection task are bidirectionally associated by using the partition filtering mechanism. First, a linear layer neural network is used to realize the aspect gate and the opinion gate of the LSTM neural network. Then, the gated mechanism is used to divide each time step unit into the aspect term recognition task partition , the review opinion detection task partition and the shared task partition . Finally, the filtering mechanism is used to filter the irrelevant information to obtain the partition filtering information . The bidirectional association of the aspect word recognition task and the comment opinion detection task by using the partition filtering mechanism comprises the following steps: S41, aspect gate and opinion gate The information distribution of the aspect term recognition task and the review opinion detection task is controlled respectively, and the neural units of each time step are divided into aspect term recognition task partitions , review opinion detection task partitions and shared task partitions The calculation methods of the aspect gate and the opinion gate are as follows: wherein, represents a computation of cumulative maximum, represents a linear transformation of a standard, represents a first overall class enhancement vector representation, represents a hidden layer vector representation of a previous time step; S5, the probability distribution score vector between each word pair is calculated by using the double affine deep attention mechanism, and the probability distribution score vector is filled into each word pair cell of the sentiment triple unified labeling space two-dimensional table; S6, adding symmetry constraints to the uniform labels in the emotion triplets uniform label space two-dimensional table and implicit constraints ; S7, the sentiment triple unified labeling space joint decoding framework is used to search the square representing the aspect word and the comment opinion and the rectangle representing the sentiment polarity in the two-dimensional table, first, the property that the aspect word or the evaluation object adjacent row or column labels are consistent is used to determine the boundary of the information of the two, second, the property that the square is symmetric about the diagonal line is used to decode the aspect word or the comment opinion, and finally, the sentiment polarity of the rectangle structure aligned between the aspect word and the comment opinion is searched by using the detected aspect word and the comment opinion; S8, the comment text aspect word sentiment triple is constructed, the advantages and disadvantages of the aspect word sentiment evaluation under each category and the causes are aggregated, and the overall comment text sentiment triple is induced to reflect the overall evaluation result, and feedback information is automatically generated according to the user's query condition; The construction of the sentiment triple unified labeling space in the step S1 comprises the following steps: S11, obtaining the start position and the end position of the aspect word in the review text and the sentiment polarity of the corresponding aspect word , the start position and the end position of the opinion word ;​ S12, obtain the category information between the aspect words and the comment opinions in the comment text, and obtain category information by statistical analysis, defined as ; S13, the obtained Based on the category information, the aspect words, the comment opinion tags and the sentiment polarity are marked, the marking mode of the aspect words is defined as , The marking mode of the comment opinion is , The joint marking mode of the sentiment polarity is , , , Indicates that there is no association between the word pairs; S14, fill the obtained aspect word label, comment opinion label and emotion polarity joint label into the respective cells of the table to represent the information category relationship between the word pairs, thereby constructing a unified sentiment triple label space, wherein represents the length of the comment text . ​​ 2. The multi-class table filling based sentiment triple generation method according to claim 1, characterized in that, The step S3 utilizes a multi-class multi-head attention mechanism to respectively learn a category-enhanced vector representation associated with the aspect word of the category to which the comment belongs and the associated category-enhanced vector representation of the opinion of the comment Specifically comprises the following steps: S31, further acquire the deep-level semantic information of the text context of each time step by using the LSTM neural network model The detailed calculation method is as follows: wherein, , are trainable parameters, denotes an activation function, , , denote the input gate, output gate and forget gate, respectively, denotes the cell state at the current time step; denotes the cell state at the previous time step; denotes the cell state update value; S32, the Bert output vector representation and the hidden layer vector output at the previous time step When inputting as a multi-class multi-head attention mechanism module, first and Dot product to obtain the semantic similarity between each category and the aspect word or the comment opinion Then and Dot product to obtain the aspect word category or comment opinion category enhanced vector representation Finally, the hidden layer output vector of the LSTM neural network model is spliced with the category enhanced vector representation to form the final vector representation of the unit time step , as shown in the following formula: wherein, represents an activation function, represents a word vector dimension of a Bert output, represents a calculation manner of an attention mechanism, , , represents a category of a text description aspect word or a comment opinion, represents a key-value pair associated with a category, and is specifically as shown in the following formula: wherein represents activation function; S33, obtaining the aspect word and the comment opinion category enhanced vector representation of the entire text sequence, respectively , wherein , and splicing the aspect word and the comment opinion category enhanced vector to obtain the final overall category enhanced vector representation .

3. The multi-class table filling based sentiment triple generation method according to claim 1, characterized in that, The bidirectional association of the aspect word recognition task and the comment opinion detection task by using the partition filtering mechanism in the step S4 further comprises the following steps: S42, further calculate the current time step information representation : wherein, represents an activation function; S43、computing the aspect word recognition task partition corresponding to the historical time step , the review opinion detection task partition , and the sharing task partition The vector information representation is specifically calculated as shown in the following formula: wherein, represents a corresponding element multiplication operator, , The calculation method of the aspect gate and the viewpoint gate is consistent with the calculation method in step S41. S44, calculate the aspect term recognition task partition corresponding to the current time step , the review opinion detection task partition and the sharing task partition The vector information representation is calculated as follows: S45, adding the partition vector of the current time step and the partition vector of the previous time step to integrate into the overall partition information representation of the current time step , and The specific calculation is as follows: S46, using the divided aspect word to identify the task partition , the review opinion detection task partition , and the sharing task partition , the different superimposed modes between them realize the reservation and filtering of information, for this, the memory storage units of the aspect word identification task partition, the review opinion detection task partition and the sharing task partition are defined as , and , the specific calculation method is as shown in the following formula: ， ， S47, splicing the three partition memory storage units to obtain a cell state vector representation of the next time step and the hidden layer vector representation The specific calculation is as follows: wherein denotes a standard linear transformation; S48、Finally, in each time step vector representation Aspects word recognition and review opinion detection are generated by partition filtering information based on splicing .

4. The multi-class table filling based sentiment triple generation method according to claim 1, characterized in that, The calculation of the probability distribution score vector between each word pair by using the double affine deep attention mechanism in the step S5 and the filling of the probability distribution score vector into each word pair cell of the sentiment triple unified labeling space two-dimensional table comprises the following steps: S51, two multi-layer MLP neural network models are used to predict the head and tail parts of each word, and the specific calculation method is as shown in the following formula: S52, score vector representation of each word pair is calculated using the bi-affine deep attention mechanism model as follows : wherein, denotes a bi-affine transformation, and are model weights, denotes an offset; S53、First, the fractional vector is represented As The input to the function predicts a probability distribution over the labels in the word pair as follows: The probability distribution for each word pair is then populated into a two-dimensional table of ​ Finally, the probability distribution of the predicted label and the real label are used to calculate the overall loss value according to the following formula: wherein is the true label.

5. The multi-class table filling based sentiment triple generation method according to claim 1, characterized in that, adding symmetry constraints to the uniform labels in the emotion triad uniform label space two-dimensional table and implicit constraints The steps are as follows: S61, the annotation structure of the aspect words and the comment opinion words are all about the square which is symmetric about the diagonal, and therefore the loss function of the symmetry constraint is defined as : wherein, represents the probability of each word in a sentence for all possible tags stacking; S62、In the sentiment triple, the sentiment polarity of the aspect word is certainly related to the aspect word and the review opinion. To this end, the loss function of the implicit constraint is defined as : wherein, represents the probability of each word in a sentence for all possible tags stacked.

Images