Sentiment causal pair extraction method, medium and device for product reviews
By constructing an emotional causal extraction model, using emotional dictionary and abstract semantic representation tools, combining BERT and graph convolutional networks, clause vector representation is generated, which solves the problem of insufficient accuracy of emotional causal extraction in the existing technology, and improves the product The effect of emotional causality in the comments on extraction.
Patent Information
- Application Number
- CN202411324378.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2044-09-23
AI Technical Summary
The existing emotional causal extraction method has shortcomings in capturing the different emotions types and document semantic structure information of the clause, resulting in a decrease in the accuracy of emotional and cause information, affecting the model extraction effect.
Using an emotional causal pair extraction method for product reviews, the emotional causal pair extraction model is constructed by obtaining clauses and using external emotional dictionary and abstract semantic representation tools. Combining BERT and graph convolution networks, the context and emotions of clauses are generated. Enhanced and semantic structure vector representation, and finally generates the score of clause pairs by matching the predictor to improve the accuracy of the decimation.
It improves the accuracy and reliability of emotional causal pair extraction, enhances the model's extraction effect of cause clauses, and provides more accurate knowledge support for practical applications.
Smart Images
Figure CN118839014B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep learning and sentiment analysis, and more specifically, to a method, medium and device for extracting sentiment causal pairs for product reviews. Background Art
[0002] Emotion-Cause Pair Extraction (ECPE) is a task that aims to extract both emotional information and corresponding cause information from text without relying on pre-labeled emotional information. This task plays an important role in real-world scenarios. For example, in terms of product consumption, ECPE can help companies understand market evaluations, analyze the reasons for negative reviews, and perform targeted optimizations. In terms of corporate decision-making, analyzing feedback suggestions from various platforms can provide effective auxiliary decision-making information, thereby helping companies make correct decisions.
[0003] Current ECPE methods can be mainly divided into two-stage methods and end-to-end methods.
[0004] The two-stage method first extracts the clauses containing emotions and the corresponding cause clauses, and then uses the Cartesian product to combine all possible clause pairs and screen out the final emotional cause-effect pairs; the two-stage model Indep proposed by Xia et al. first identifies and extracts the clauses containing emotions and the corresponding cause clauses; then, by implementing the Cartesian product operation, all possible clause pairs are combined; finally, the most reasonable emotional cause-effect pairs are screened out from these combinations as the final result; specifically, in the first step, Indep first uses the bidirectional long short-term memory network (Bi-LSTM) technology to learn the vector representation of each word in the document, and then introduces the attention mechanism to further derive the vector representation of each clause, and then uses two Bi-LSTMs to obtain the emotional Specific clause representation and cause-specific clause representation, and finally predict the sentiment clause and cause clause respectively through two classifiers; in the second step, the clause sets extracted in the first step are combined in pairs using the Cartesian product operation to construct a complete set of all possible causal clause pairs, and then the classifier is used to screen these candidate pairs to determine the final sentiment causal pairs; although the two-stage model has achieved good results in the sentiment causal pair extraction task, it still has the following shortcomings: ① The error in the first step may affect the performance of the second step and cause error propagation; ② The model is difficult to jointly optimize; The end-to-end method refers to training the entire system as a single model, and the model directly learns and predicts end-to-end from the original input data to the final output result in one step.
[0005] Due to the problems of error propagation and difficulty in optimization in the two-stage model, the current mainstream ECPE methods are mainly based on end-to-end. Most methods first use a pre-trained model to generate a vector representation for each clause in the document, and then use various deep learning models to learn the relationship between clauses to obtain a clause vector representation that takes into account the contextual relationship; finally, the vector representation of the clause pair is obtained by combining the vector representations of the clauses, and a classifier is used to determine whether there is a causal relationship between the clause pairs; Cheng et al. proposed a network composed of a pair of complementary sub-network units, namely the sentiment sub-network and the cause sub-network; these two sub-networks each contain two key components: a clause representation learning module and a local pair search module; in this model, the clause representation learning module is responsible for understanding and extracting the sentiment features or causal relationships contained in each clause; while the local pair search module is dedicated to constructed a graph convolutional network model, which aims to simulate three different types of relational dependencies between local adjacent candidate pairs, thereby helping to more effectively capture and analyze the contextual information of the text; Wei et al. extracted sentiment causal pairs from a sorting perspective, and proposed a method that emphasizes modeling between clauses to perform end-to-end extraction; this method first adopted a graph attention mechanism to learn and construct vector representations of clauses, thereby modeling the mutual connections between clauses and enhancing the extraction effect by revealing the potential correlation between the two clauses; subsequently, by learning the joint representation of clause pairs and sorting these clause pairs, effective extraction of sentiment causal pairs is achieved; the end-to-end method trains the entire system as a single model, inputs the original data, and the model directly outputs the final judgment result in one step.
[0006] Existing ECPE methods mainly focus on learning universal clause representations by exploring interactions between words or clauses to improve the extraction performance of the model, but they ignore the different sentiment types of clauses and the semantic structure information of the document, which is crucial for accurately extracting sentiment and corresponding cause information; this will affect the accuracy of the model in capturing sentiment and cause clues, thereby reducing the extraction effect of sentiment causal pairs. Summary of the invention
[0007] The purpose of the present invention is to provide a method, medium and device for extracting sentiment causal pairs for product reviews, which can improve the accuracy and reliability of extracting sentiment causal pairs.
[0008] The present invention provides a method for extracting sentiment causal pairs for product reviews, comprising the following steps: S1: obtaining existing product review data, performing clause segmentation and structured processing on the existing product review data to obtain clauses and training data; S2: obtaining sequential indexes based on the clauses and the training data using an external sentiment dictionary; S3: obtaining a directed acyclic graph based on the clauses using an abstract semantic representation tool; S4: constructing a sentiment causal pair extraction model based on the training data, the sequential index and the directed acyclic graph, the sentiment causal pair extraction model being used for extracting sentiment causal pairs for product reviews; S5: inputting target product review data into the sentiment causal pair extraction model to obtain clause pairs and their scores for becoming sentiment causal pairs, and taking clause pairs whose ranking numbers are less than a preset number as sentiment causal pairs.
[0009] Furthermore, step S1 of the above-mentioned sentiment causal pair extraction method for product reviews specifically includes: obtaining existing product review data, performing clause segmentation and structural processing on the existing product review data, and obtaining clauses and training data, such as formula:
[0010] ,
[0011] ,
[0012] in, is training data in the form of documents consisting of multiple clauses, Indicates clauses, Representation Document Length, Indicates The first Words, Expression Clause Length.
[0013] Furthermore, the sentiment dictionary of the above sentiment causal pair extraction method for product reviews contains six sentiment types: happiness, sadness, fear, anger, embarrassment and surprise, and assigns sequential indexes from 1 to 6 to represent these six sentiment types in turn; index 0 is designated as "neutral", indicating the lack of a specific sentiment type.
[0014] Furthermore, step S3 of the above-mentioned method for extracting sentiment causal pairs for product reviews specifically includes: according to the clauses, using an abstract semantic representation tool, obtaining a directed acyclic graph, such as formula:
[0015] ,
[0016] ,
[0017] ,
[0018] in, for The parsed directed acyclic graph, Represents a collection of nodes. Represents a set of relationships between nodes. Expression Clause The phrase or word node consisting of the a-th word to the b-th word in .
[0019] Furthermore, step S4 of the above-mentioned sentiment causal pair extraction method for product reviews specifically includes: S41: constructing a triple clause encoder, the triple clause encoder is used to obtain the final clause vector representation; S42: constructing a matching predictor, the matching predictor is used to obtain candidate clause pairs and their scores based on the final clause vector representation; S43: obtaining a causal pair extraction model based on the triple clause encoder and the matching predictor; using training data, sequential indexes and directed acyclic graphs to train the causal pair extraction model to obtain a sentiment causal pair extraction model, which is used for sentiment causal pair extraction for product reviews.
[0020] Furthermore, step S41 of the above-mentioned method for extracting sentiment causal pairs for product reviews specifically includes: S411: using BERT to build a pre-trained model, the pre-trained model is used to obtain the context clause vector representation, such as formula:
[0021] ,
[0022] in, For Clause The context vector representation of Indicates clauses, Expression Clause Length, For Clause The qth component of the word-level vector representation of the context obtained by encoding the BERT model; S412: constructing a sentiment detection module, the sentiment detection module is used to obtain the sentiment enhancement clause vector representation, such as the formula:
[0023] ,
[0024] in, For Clause The sentiment enhancement vector representation is The corresponding vector is obtained by mapping the index 0 corresponding to the seven sentiment types in the sentiment dictionary. The index corresponding to the seven sentiment types in the sentiment dictionary (0~6) are mapped to the corresponding vectors, For Clause The number of all non-zero indices, Represents the first of all non-zero indices indexes; S413: using the graph convolutional network, construct a graph module based on the semantic structure, and the graph module based on the semantic structure is used to obtain the vector representation of the semantic structure clause, such as the formula:
[0025] ,
[0026] ,
[0027] ,
[0028] ,
[0029] ,
[0030] in, For Clause The semantic structure vector representation of Expression Clause Length, For Clause The semantic structure vector representation of the qth word, For Clause Any word in The semantic structure vector representation of Expression Clause Middle The context vector representation of the word, represents the indicator function, Expression Clause The phrase or word node consisting of the ath word to the bth word in For Node The final node representation of and is a trainable matrix, For graph convolutional network Layer Node exist The hidden state of the layer, ReLU(.) is the activation function, Represents various relationship types of AMR graphs, Indicates Nodes connected by relationship types Neighbors, is the normalization constant, is a trainable matrix, For Clause Nodes in In the clause The vector representation in ;
[0031] S414: According to the pre-trained model, the sentiment detection module and the graph module based on the semantic structure, a triple clause encoder is obtained, and the triple clause encoder is used to obtain the final clause vector representation, such as the formula:
[0032] ,
[0033] in, For Clause The final vector representation of For Clause The semantic structure vector representation of For Clause The sentiment enhancement vector representation is For Clause The context vector representation of .
[0034] Furthermore, step S42 of the above-mentioned sentiment causal pair extraction method for product reviews specifically includes: constructing a matching predictor, which is used to obtain candidate clause pairs and their scores according to the final clause vector representation, such as formula:
[0035] , , ,
[0036] in, Candidate clause pair The score of , tanh() is the hyperbolic tangent function, Candidate clause pair The vector representation of and is a trainable weight matrix and bias vector, ReLU(.) is the activation function, and Respectively represent clauses AND clause The vector representation of Expressing clause pairs The relative position vector represents and is a trainable weight matrix and bias vector, and the relative position vector representation can be obtained by random initialization; represents the set of all candidate clause pairs, For Clause AND clause The candidate clause pairs are composed of Represents a document.
[0037] Furthermore, step S43 of the above-mentioned method for extracting sentiment causal pairs for product reviews specifically includes: obtaining a causal pair extraction model according to a triple clause encoder and a matching predictor; constructing a loss function, and training the causal pair extraction model using the loss function, training data, sequential indexes, and directed acyclic graphs to obtain a sentiment causal pair extraction model, and the sentiment causal pair extraction model is used for extracting sentiment causal pairs for product reviews, as shown in the formula:
[0038] ,
[0039] in, is the loss function , Represents a document, Is a clause pair Is it the true value of the sentiment causal pair? Is a clause pair Whether it is the predicted value of the sentiment causal pair.
[0040] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned method for extracting sentiment causal pairs for product reviews.
[0041] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the above-mentioned method for extracting emotional causal pairs for product reviews are implemented.
[0042] Implementing the method, medium and device for extracting sentiment causal pairs for product reviews provided by the present invention has the following beneficial effects:
[0043] The present invention aims at obtaining the sentiment type information of document clauses, and uses an external sentiment dictionary to identify words or phrases with sentiment polarity and determine the sentiment type of the clause. Through this method, the probability of successfully matching clauses with similar sentiment types is greater, so that sentiment causal pairs can be extracted more accurately.
[0044] The present invention aims at obtaining the semantic structure information of document clauses. By using an abstract semantic representation tool, the semantic structure information of each clause is obtained, and some association information within and between clauses is provided to the model, thereby enhancing the model's extraction effect on cause clauses.
[0045] The present invention designs an emotion detection module and a graph module based on semantic structure, which effectively model the emotion type information and semantic structure information respectively, thereby generating the emotion-enhanced clause representation and semantic structure representation of the clause; through this method, the model can more accurately extract the emotion causal pairs, and provide more accurate and reliable knowledge support for the fields of product optimization and decision support in practical applications;
[0046] The present invention designs a pairing predictor, which generates a clause pair vector representation using a clause vector representation and an inter-clause vector representation, and scores the possibility of each clause pair becoming a sentiment causal pair through a classifier, thereby improving the reliability of the model in extracting sentiment causal pairs and providing a solid foundation for real-world product review analysis. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:
[0048] Figure 1 It is a flow chart of the method for extracting sentiment causal pairs for product reviews provided by the present invention;
[0049] Figure 2 It is a schematic diagram of the results obtained through AMR analysis provided by the present invention;
[0050] Figure 3 This is a structural diagram of the automatic extraction model of sentiment causal pairs of product reviews provided by the present invention;
[0051] Figure 4 This is a flow chart for realizing automatic extraction of sentiment and reason information from product review data provided by the present invention;
[0052] Figure 5 It is a structural block diagram of the computer device provided by the present invention. DETAILED DESCRIPTION
[0053] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.
[0054] Figure 1 A schematic diagram of the method for extracting sentiment causal pairs from product reviews in this embodiment is shown; in this embodiment, the method for extracting sentiment causal pairs from product reviews includes the following steps:
[0055] S1: Obtain existing product review data, perform clause segmentation and structural processing on the existing product review data to obtain clauses and training data;
[0056] Specifically, the above step S1 specifically includes: obtaining existing product review data, performing clause segmentation and structural processing on the existing product review data, and obtaining clauses and training data, such as formula:
[0057] ,
[0058] ,
[0059] in, is training data in the form of documents consisting of multiple clauses, Indicates clauses, Representation Document Length, Indicates The first Words, Expression Clause Length;
[0060] S2: Based on the clauses and training data, use the external sentiment dictionary to obtain the sequential index;
[0061] Specifically, the above emotion dictionary contains six emotion types: happy, sad, fearful, angry, embarrassed, and surprised, and assigns sequential indexes from 1 to 6 to represent these six emotion types in turn; index 0 is designated as "neutral", indicating the lack of a specific emotion type;
[0062] S3: Based on the clauses, we use the abstract semantic representation tool to obtain a directed acyclic graph;
[0063] Specifically, the above step S3 specifically includes: according to the clauses, using an abstract semantic representation tool, obtaining a directed acyclic graph, such as formula:
[0064] ,
[0065] ,
[0066] ,
[0067] in, for The parsed directed acyclic graph, Represents a collection of nodes. Represents a set of relationships between nodes. Expression Clause The phrase or word node consisting of the ath word to the bth word in ;
[0068] S4: Based on the training data, sequential index and directed acyclic graph, a sentiment causal pair extraction model is constructed. The sentiment causal pair extraction model is used to extract sentiment causal pairs for product reviews.
[0069] Specifically, the above step S4 specifically includes:
[0070] S41: construct a triple clause encoder, which is used to obtain the final clause vector representation;
[0071] Specifically, the above step S41 specifically includes:
[0072] S411: Use BERT to build a pre-trained model, which is used to obtain the context clause vector representation, such as the formula:
[0073] ,
[0074] in, For Clause The context vector representation of Indicates clauses, Expression Clause Length, For Clause The qth component of the word-level vector representation of the context encoded by the BERT model;
[0075] S412: Construct an emotion detection module, which is used to obtain a vector representation of the emotion enhancement clause, such as the formula:
[0076] ,
[0077] in, For Clause The sentiment enhancement vector representation is The corresponding vector is obtained by mapping the index 0 corresponding to the seven sentiment types in the sentiment dictionary. The index corresponding to the seven sentiment types in the sentiment dictionary (0~6) are mapped to the corresponding vectors, For Clause The number of all non-zero indices, Represents the first of all non-zero indices indexes;
[0078] S413: Using the graph convolutional network, a graph module based on the semantic structure is constructed. The graph module based on the semantic structure is used to obtain the vector representation of the semantic structure clause, such as the formula:
[0079] ,
[0080] ,
[0081] ,
[0082] ,
[0083] ,
[0084] in, For Clause The semantic structure vector representation of Expression Clause Length, For Clause The semantic structure vector representation of the qth word, For Clause Any word in The semantic structure vector representation of Expression Clause Middle The context vector representation of the word, represents the indicator function, Expression Clause The phrase or word node consisting of the ath word to the bth word in For Node The final node representation of and is a trainable matrix, For graph convolutional network Layer Node exist The hidden state of the layer, ReLU(.) is the activation function, Represents various relationship types of AMR graphs, Indicates Nodes connected by relationship types Neighbors, is the normalization constant, is a trainable matrix, For Clause Nodes in In the clause The vector representation in ;
[0085] S414: According to the pre-trained model, the sentiment detection module and the graph module based on the semantic structure, a triple clause encoder is obtained, and the triple clause encoder is used to obtain the final clause vector representation, such as the formula:
[0086] ,
[0087] in, For Clause The final vector representation of For Clause The semantic structure vector representation of For Clause The sentiment enhancement vector representation is For Clause Context vector representation of ;
[0088] S42: construct a matching predictor, where the matching predictor is used to obtain candidate clause pairs and their scores according to the final clause vector representation;
[0089] Specifically, the above S42 specifically includes: constructing a matching predictor, which is used to obtain candidate clause pairs and their scores according to the final clause vector representation, such as the formula:
[0090] ,
[0091] ,
[0092] ,
[0093] in, Candidate clause pair The score of , tanh() is the hyperbolic tangent function, Candidate clause pair The vector representation of and is a trainable weight matrix and bias vector, ReLU(.) is the activation function, and Respectively represent clauses AND clause The vector representation of Expressing clause pairs The relative position vector represents and is a trainable weight matrix and bias vector, and the relative position vector representation can be obtained by random initialization; represents the set of all candidate clause pairs, For Clause AND clause The candidate clause pairs are composed of Represents a document;
[0094] S43: According to the triple clause encoder and the matching predictor, a causal pair extraction model is obtained; the causal pair extraction model is trained using the training data, the sequential index and the directed acyclic graph to obtain the sentiment causal pair extraction model, which is used for sentiment causal pair extraction for product reviews;
[0095] Specifically, the above step S43 specifically includes: obtaining a causal pair extraction model according to a triple clause encoder and a matching predictor; constructing a loss function, and training the causal pair extraction model using the loss function, training data, sequential index and directed acyclic graph to obtain an emotional causal pair extraction model. The emotional causal pair extraction model is used for emotional causal pair extraction for product reviews, such as formula:
[0096] ,
[0097] in, is the loss function , Represents a document, Is a clause pair Is it the true value of the sentiment causal pair? Is a clause pair Whether it is the predicted value of the emotional causal pair;
[0098] S5: Input the target product review data into the sentiment causal pair extraction model to obtain the clause pairs and their scores for becoming sentiment causal pairs, and take the clause pairs whose ranking numbers are less than the preset numbers as sentiment causal pairs.
[0099] In some embodiments, the above-mentioned sentiment causal pair extraction method for product reviews can also be implemented in the following way.
[0100] In this embodiment, the sentiment causal pair extraction method for product reviews is to represent the product review data in the form of a document of multiple clauses, and effectively model each clause from the perspectives of context, sentiment type and semantic structure, thereby generating a context vector representation, a sentiment enhancement vector representation and a semantic structure vector representation, and then fuse the vector representations of these three perspectives to obtain the vector representation of the clause, and finally obtain the vector representation of the clause pair through the vector representation of the clause and the vector representation of the relative position between the clauses, and calculate the score of each clause pair, thereby achieving the purpose of automatic extraction of sentiment causal pairs; specifically, Figure 4 As shown, proceed as follows:
[0101] Step 1: Obtain existing product review data, split each review data into clauses, and structure it to form a document consisting of multiple clauses. As training data, for example, documents Contains 4 clauses, ( )He told us that since he left school, )His classmates and tutors gave him a lot of help with his homework, ( He was very touched. ) and says he will repay them; the sentiment clause of this document is , the corresponding reason clause is , so the emotional causal pair is ;in, Represents a document, Indicates clauses, Representation Document The length of Clauses ,in, Indicates The first Words, Expression Clause Length;
[0102] Step 2: Introduce an external emotion dictionary, which contains six emotion types: happy, sad, fearful, angry, embarrassed, and surprised, and assigns sequential indexes from 1 to 6 to represent these six emotion types; in addition, index 0 is designated as "neutral", indicating the lack of a specific emotion type;
[0103] For example, the clause and The sentiment type is "neutral", the corresponding index is 0, and the clause Contains the sentiment word "help", whose sentiment type is "happy", and the corresponding index is 1, clause Contains the emotional word "touched", whose emotional type is "happy", and the corresponding index is 1;
[0104] Step 3: Introduce the Abstract Meaning Representation (AMR) tool to parse each clause into a directed acyclic graph consisting of nodes and relationships labeled according to concepts. Specifically, through the AMR parser, a clause Can be parsed as an AMR graph ,in, Represents a collection of nodes. Expression Clause Zhongcong arrive The phrases or word nodes that make up the A collection representing the relationships between nodes;
[0105] For example, for the clause "the boy hopes the girl will believe him", the result obtained after AMR parsing is as follows Figure 2As shown; for the clause "the boy hopes that the girl believes him", after AMR parsing, the nodes are "boy", "hope", "girl", "believe" and "he", and there is a relationship "arg0" between "boy" and "hope", a relationship "arg0" between "boy" and "believe", a relationship "arg0" between "girl" and "believe", a relationship "arg0" between "believe" and "he", and a relationship "arg1" between "believe" and "hope";
[0106] Step 4: Construct a triple clause encoder. This part uses BERT, sentiment detection module and semantic structure-based graph module to encode the input document. These three parts generate context-combined vector representation, sentiment-enhanced vector representation and semantic structure vector representation for the clause respectively, which enables the model to generate more discriminative clause vector representation; Figure 3 The figure shows the structure of the model for automatic extraction of sentiment causal pairs of product reviews;
[0107] Step 4.1 generates a context vector representation for the clause:
[0108] Step 4.1.1 Pair Clauses , insert a [CLS] tag at the beginning of the clause and a [SEP] tag at the end of the clause, and you get ,Will As the input of the pre-trained model BERT, we get the word-level vector representation of the context , for clauses The word-level vector representations except [CLS] and [SEP] in [CLS] are average pooled, and the clause is obtained using formula (1) The vector representation of the context :
[0109] (1)
[0110] In formula (1), , , is the hidden layer size of BERT;
[0111] Step 4.1.2 AND Clause Same as above, get the context vector representation of all clauses ;
[0112] Step 4.2 Design a sentiment detection module to generate sentiment-enhanced vector representations for clauses:
[0113] Step 4.2.1 The indexes corresponding to the seven sentiment types in the sentiment dictionary (0~6) are mapped to the corresponding vectors respectively , for the clause , using the sentiment dictionary to identify the words or phrases in the clause and assign corresponding sentiment type indexes, in order to determine the clause Sentiment Enhanced Vector Representation , follow the following rules: (1) If the clause If all the indices in are 0, then the sentiment enhancement vector represents ; (2) If clause If there is a non-zero index in , the sentiment enhancement vector representation is obtained by average pooling , as shown in formula (2):
[0114] (2)
[0115] In formula (2), Is Clause The number of all non-zero indices, Represents the first of all non-zero indices indexes;
[0116] For example, for the clause "Seeing the family reunion made her cry", there are emotional words "reunion", "leaving her hometown", "crying", and their corresponding emotional types are "happy", "sad" and "sad", and the corresponding indexes are 1, 2 and 2 respectively. Then the emotional enhancement vector representation of this clause should be 3 index vectors: , , The average value of
[0117] Step 4.2.2 AND Clause Same as above, get the sentiment enhancement vector representation of all clauses ;
[0118] Step 4.3 Design a graph module based on semantic structure to generate semantic structure vector representation for clauses:
[0119] Step 4.3.1 For the clause Nodes in , through formula (3) we can get its The vector representation in :
[0120] (3)
[0121] For example, for the clause “the boy hopes the girl will believe him”, the vector representation of the node “boy” should be the average of the contextual representations of the words “male” and “female”;
[0122] This module uses a M A graph convolutional network with stacked layers is used to simulate the interaction between different nodes through edges of different relationship types; Layer Node , exist Hidden state of a layer It can be obtained from formula (4):
[0123] (4)
[0124] In formula (4), Represents various relationship types of AMR graphs, Indicates Nodes connected by relationship types Neighbors, is a normalizing constant, and is a trainable matrix;
[0125] Using formula (5) we can get the node The final node representation :
[0126] (5)
[0127] In formula (5), and is a trainable matrix;
[0128] Step 4.3.2 For Clause Any word in , using formula (6) to get its corresponding semantic structure vector representation :
[0129] (6)
[0130] In formula (6), represents the indicator function, Expression Clause Middle The context vector representation of the word;
[0131] Using formula (7), we can get the clause The semantic structure vector representation of:
[0132] (7)
[0133] Step 4.3.3 AND Clause Same as above, get the semantic structure vector representation of all clauses ;
[0134] Step 4.4 Use formula (8) to get the clause The final vector representation of is:
[0135] (8)
[0136] And Clause Same as above, get the final vector representation of all clauses ;
[0137] Step 5: Build a pairing predictor to generate a vector representation of a clause pair through the clause vector representation and the relative position information between clauses. This enables the model to generate more accurate vector representations of clause pairs and more accurately predict the final sentiment causal pair.
[0138] Step 5.1 For Documentation , consider all combinations of clauses ,in represents the set of all candidate clause pairs, and means clause AND clause The candidate clause pairs formed;
[0139] Among them, the candidate clause pair Vector representation of It can be obtained from formula (9):
[0140] (9)
[0141] In formula (9), and are trainable weight matrices and bias vectors, and Respectively represent clauses AND clause The vector representation of Expressing clause pairs The relative position vector representation of , which can be obtained by random initialization;
[0142] Step 5.1.2 Using formula (10) candidate clause pair Score :
[0143] (10)
[0144] In formula (9), and are trainable weight matrices and bias vectors;
[0145] Step 5.1.3: Compare with clauses Same, get the scores of all clause pairs;
[0146] For example, for the document :( )He told us that since he left school, )His classmates and tutors gave him a lot of help with his homework, ( He was very touched. ) and says he will repay them; the sentiment clause of this document is , the corresponding reason clause is , emotional causality ; For clause pairs , its score should be higher than all other clause pairs;
[0147] Step 6: The sentiment causal pair extraction model is constructed by the triple clause encoder and the pairing predictor, and the sentiment causal pair extraction model is trained using the back propagation algorithm based on the product review training data, and the loss function shown in formula (11) is calculated at the same time. To update the model parameters, when the loss function When the value keeps decreasing until convergence, the optimal emotion causal pair extraction model is obtained:
[0148] (11)
[0149] In formula (11), Is a clause pair Whether it is the true value of the sentiment causal pair;
[0150] Step 7: Input the existing product review data into the optimal sentiment causal pair extraction model, so as to output the clause pairs generated by the model as the scores of the sentiment causal pairs, and output the clause pairs with the highest scores as the sentiment causal pairs, so as to realize the automatic extraction of sentiment and cause information in the product review data; Figure 4 Shown is a flowchart for automatically extracting sentiment and reason information from product review data.
[0151] In order to verify the effectiveness of the sentiment causal pair extraction method for product reviews in this embodiment, this embodiment conducts experiments on a benchmark dataset for sentiment causal pair extraction; the dataset contains a total of 1945 Chinese documents. Table 1 summarizes the detailed statistical information of the dataset; when conducting the experiment, the experiment was repeated 15 times, and the average results were taken on the main task: sentiment causal pair extraction (ECPE) and two auxiliary tasks: sentiment clause extraction (EE) and cause clause extraction (CE);
[0152] Table 1: Statistics of the dataset
[0153]
[0154] To evaluate the performance of our methods, this section uses three common metrics: precision ( P)、Recall( R )and Fraction:
[0155] (12)
[0156] (13)
[0157] (14)
[0158] Among them, the total number of predicted pairs ( ) represents the number of sentiment causal pairs predicted by the method, and the total number of pairs annotated in the dataset ( ) represents the number of sentiment causal pairs contained in the dataset, and the number of correct pairs predicted ( ) represents the number of causal pairs that are both predicted and labeled as sentiment;
[0159] The method of this embodiment is compared with other methods in the field of emotion causal pair extraction to verify whether the method of this embodiment can effectively extract emotion information and locate cause information; for this purpose, this embodiment selects 7 methods, namely Index, ECPE-2D, RankCP, MGSAG, MaCa, PBJE and JCB, for comparative experiments, and the experimental results are shown in Table 2;
[0160] Table 2: Comparison of experimental results
[0161]
[0162] (1) Indep is a two-step method that first extracts sentiment clauses and cause clauses separately, then pairs the extracted sentiments and causes, and selects the final sentiment-cause pairs;
[0163] (2) ECPE-2D represents emotion-cause pairs through a 2D representation scheme and integrates pair representation and interaction and prediction into a joint framework;
[0164] (3) RankCP is a unified framework for solving ECPE tasks from a ranking perspective;
[0165] (4) MGSAG aims to alleviate the location bias problem by jointly introducing fine-grained and coarse-grained semantic features;
[0166] (5) MaCa treats the extraction of sentiment causal pairs as a process of sequence modeling;
[0167] (6) PBJE generates pair and clause features simultaneously by joint feature encoding to model the causal relationship in clauses;
[0168] (7) JCB proposed a boundary adjustment model with joint constraint learning;
[0169] The effectiveness of the model is mainly based on its performance on the ECPE task. The performance of the two auxiliary tasks, EE and CE, is considered secondary. By analyzing the experimental results, the following conclusions can be drawn:
[0170] (1) For the ECPE task, it is obvious that the method of this embodiment The value is 0.89% higher than that of JCB. Through experimental analysis, it is found that this improvement mainly comes from P Value and R The value of the balance increase; compared with RankCP, R The value is increased by 2.8%; the experimental results show that it is effective to extract useful information of emotional causal pairs by using emotional type information and semantic structure; although P The values may not be optimal, but are still competitive with the state-of-the-art methods;
[0171] (2) For the auxiliary task CE, the method of this embodiment shows the best performance compared with other baseline methods; compared with JCB and PBJE, the method of this embodiment has The values were increased by 0.83% and 1.5% respectively, compared with RankCP. R The value was increased by 2.5%. In the EE task, although the method of this embodiment did not achieve the best score, it still achieved better results.
[0172] (3) From the experimental data, we can further analyze that the sentiment detection module and the semantic structure-based graph module cooperate with each other to provide the model with clues of sentiment and semantic structure respectively, thereby facilitating the extraction of sentiment clauses and cause clauses.
[0173] Table 3: Comparison of ablation experiment results
[0174]
[0175] In order to verify the effects of the emotion detection module (ED) and the semantic structure-based graph module (SGM) in the method of this embodiment, this embodiment conducts ablation experiments on these two modules respectively. The results are shown in Table 3. It can be observed that the model without the emotion detection module leads to a decrease in performance. The value decreased by 1.49%, while P Value and R The values dropped by 1.46% and 1.5% respectively. This shows the importance of using sentiment type information to enhance the model's understanding of the sentiment expressed in the text, further significantly improving the extraction performance; without a graph module based on semantic structure, The value decreased by 1.01%. P Value and R The values decreased by 1.08% and 0.88% respectively; this indicates that the introduction of semantic structure helps capture relevant relationships and facilitates the process of sentiment causal pair extraction.
[0176] The present embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned method for extracting sentiment causal pairs for product reviews; wherein the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium may also include a combination of the above-mentioned types of memories.
[0177] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the above-mentioned method for extracting sentiment causal pairs for product reviews are implemented;
[0178] like Figure 5As shown, the computer device may include: at least one processor 121, such as a CPU (Central Processing Unit), at least one communication interface 123, a memory 124, and at least one communication bus 122; wherein the communication bus 122 is used to realize the connection and communication between these components; wherein the communication interface 123 may include a display screen (Display), a keyboard (Keyboard), and the optional communication interface 123 may also include a standard wired interface and a wireless interface; the memory 124 may be a high-speed RAM memory (Random Access Memory, volatile random access memory), or a non-volatile memory (non-volatile memory), such as at least one disk memory; the memory 124 may also be at least one storage device located away from the aforementioned processor 121; wherein the memory 124 stores an application program, and the processor 121 calls the program code stored in the memory 124 to perform any of the above method steps; wherein the communication bus 122 may be a peripheral component interconnect (PCI) bus or an extended industry standard structure (Extended Industry Standard Architecture) bus. architecture, referred to as EISA) bus, etc.; the communication bus 122 can be divided into an address bus, a data bus, a control bus, etc.; for ease of representation, Figure 5Only one line is used to represent it, but it does not mean that there is only one bus or one type of bus; wherein, the memory 124 may include a volatile memory (English: volatile memory), such as a random access memory (English: random-access memory, abbreviated: RAM); the memory may also include a non-volatile memory (English: non-volatile memory), such as a flash memory (English: flash memory), a hard disk drive (English: hard disk drive, abbreviated: HDD) or a solid-state drive (English: solid-state drive, abbreviated: SSD); the memory 124 may also include a combination of the above-mentioned types of memory; wherein, the processor 121 may be a central processing unit (English: central processing unit, abbreviated: CPU), a network processor (English: network processor, abbreviated: NP) or a combination of CPU and NP; wherein, the processor 121 may further include a hardware chip; the above-mentioned hardware chip may be an application-specific integrated circuit (English: application-specific integrated circuit, abbreviated: ASIC), a programmable logic device (English: programmable logic device, abbreviated: PLD) or a combination thereof; the above-mentioned PLD may be a complex programmable logic device (English: complex programmable logic device logicdevice, abbreviated as CPLD), field-programmable gate array (English: field-programmable gate array, abbreviated as FPGA), general array logic (English: generic array logic, abbreviated as GAL) or any combination thereof; optionally, the memory 124 is also used to store program instructions; the processor 121 can call the program instructions to implement the sentiment causal pair extraction method for product reviews as in the present embodiment.
[0179] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.
Claims
1. A method for extracting sentiment causal pairs from product reviews, characterized in that: The following steps are involved: S1: Obtain existing product review data, perform clause segmentation and structural processing on the existing product review data, and obtain clauses and training data; S2: according to the clauses and the training data, using the external sentiment dictionary, obtaining a sequential index; S3: Introduce an abstract semantic representation tool to parse each clause into a directed acyclic graph consisting of nodes and relationships labeled according to concepts; S4: constructing a sentiment causal pair extraction model according to the training data, the sequential index and the directed acyclic graph, wherein the sentiment causal pair extraction model is used for sentiment causal pair extraction for product reviews; S5: Input the target product review data into the sentiment causal pair extraction model to obtain the clause pairs and their scores for becoming sentiment causal pairs, and take the clause pairs whose ranking numbers are less than the preset numbers as sentiment causal pairs; Step S4 specifically includes: S41: construct a triple clause encoder, wherein the triple clause encoder is used to obtain a final clause vector representation; S42: constructing a matching predictor, wherein the matching predictor is used to obtain candidate clause pairs and their scores according to the final clause vector representation; S43: obtaining a causal pair extraction model according to the triple clause encoder and the matching predictor; training the causal pair extraction model using the training data, the sequential index and the directed acyclic graph to obtain an emotional causal pair extraction model, wherein the emotional causal pair extraction model is used for emotional causal pair extraction for product reviews; Step S41 specifically includes: S411: Use BERT to build a pre-trained model, which is used to obtain a context clause vector representation, such as the formula: ， in, For Clause The context vector representation of Indicates clauses, Representation Clause Length, For Clause The qth component of the word-level vector representation of the context encoded by the BERT model; S412: Construct an emotion detection module, which is used to obtain a vector representation of the emotion enhancement clause, such as the formula: ， in, For Clause The sentiment enhancement vector representation is The corresponding vector is obtained by mapping the index 0 corresponding to the seven sentiment types in the sentiment dictionary. The index corresponding to the seven sentiment types in the sentiment dictionary The corresponding vectors obtained by mapping are: For Clause The number of all non-zero indices, Represents the first of all non-zero indices indexes; S413: Using a graph convolutional network, a graph module based on a semantic structure is constructed, and the graph module based on a semantic structure is used to obtain a vector representation of a semantic structure clause, such as the formula: ， ， ， ， ， in, For Clause The semantic structure vector representation of Representation Clause Length, For Clause The semantic structure vector representation of the qth word, For Clause Any word in The semantic structure vector representation of Representation Clause Middle The context vector representation of the word, represents the indicator function, Representation Clause The phrase or word node consisting of the ath word to the bth word in For Node The final node representation of and is a trainable matrix, For graph convolutional network Layer Node exist The hidden state of the layer, ReLU(.) is the activation function, Represents various relationship types of AMR graphs, Indicates Nodes connected by relationship types Neighbors, is the normalization constant, is a trainable matrix, For Clause Nodes in In the clause The vector representation in ; S414: According to the pre-trained model, the sentiment detection module and the semantic structure-based graph module, a triple clause encoder is obtained, and the triple clause encoder is used to obtain a final clause vector representation, such as formula: ， in, For Clause The final vector representation of For Clause The semantic structure vector representation of For Clause The sentiment enhancement vector representation is For Clause The context vector representation of .
2. The method for extracting sentiment causal pairs from product reviews according to claim 1 is characterized in that: Step S1 specifically includes: obtaining existing product review data, performing clause segmentation and structural processing on the existing product review data, and obtaining clauses and training data, such as formula: ， ， in, is training data in the form of documents consisting of multiple clauses, Indicates clauses, Representation Document Length, Indicates The first Words, Representation Clause Length.
3. The method for extracting sentiment causal pairs from product reviews according to claim 1 is characterized in that: The emotion dictionary contains six emotion types: happy, sad, fearful, angry, embarrassed and surprised, and assigns sequential indexes from 1 to 6 to represent these six emotion types in turn; index 0 is designated as "neutral", indicating the lack of a specific emotion type.
4. The method for extracting sentiment causal pairs from product reviews according to claim 1 is characterized in that: Step S3 specifically includes: according to the clause, using an abstract semantic representation tool, obtaining a directed acyclic graph, such as formula: ， ， ， in, for The parsed directed acyclic graph, Represents a collection of nodes. Represents a set of relationships between nodes. Representation Clause The phrase or word node consisting of the a-th word to the b-th word in .
5. The method for extracting sentiment causal pairs from product reviews according to claim 1 is characterized in that: Step S42 specifically includes: constructing a matching predictor, wherein the matching predictor is used to obtain candidate clause pairs and their scores according to the final clause vector representation, such as the formula: ， ， ， in, Candidate clause pair The score of , tanh() is the hyperbolic tangent function, Candidate clause pair The vector representation of and is a trainable weight matrix and bias vector, ReLU(.) is the activation function, and Respectively represent clauses AND clause The vector representation of Expressing clause pairs The relative position vector represents and is a trainable weight matrix and bias vector, and the relative position vector representation is obtained by random initialization; represents the set of all candidate clause pairs, For Clause AND clause The candidate clause pairs are composed of Represents a document.
6. The method for extracting sentiment causal pairs from product reviews according to claim 1, characterized in that: Step S43 specifically includes: obtaining a causal pair extraction model according to the triple clause encoder and the matching predictor; constructing a loss function, and training the causal pair extraction model using the loss function, training data, sequential index and directed acyclic graph to obtain an emotional causal pair extraction model, wherein the emotional causal pair extraction model is used for emotional causal pair extraction for product reviews, as shown in the formula: ， in, is the loss function , Represents a document, Is a clause pair Is it the true value of the sentiment causal pair? Is a clause pair Whether it is the predicted value of the sentiment causal pair.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for extracting sentiment causal pairs for product reviews as described in any one of claims 1-6 are implemented.
8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for extracting sentiment causal pairs for product reviews as described in any one of claims 1-6 are implemented.