Emotion Cause Extraction Method and System Based on Knowledge-Driven Multi-Class Classification

Abstract

The present invention discloses a method and system for extracting emotion-cause pairs based on knowledge-driven multi-classification. After tokenizing the dataset documents, the present invention calculates the emotion scores of each word in the clause and sums them up to obtain the emotion score of the clause. Then, the long short-term memory artificial neural network is used to learn the context information of the document, and the emotion-cause pairs are extracted by combining positional embedding learning and window search. The present invention combines semantic knowledge embedding, positional information learning and window search to improve the accuracy of emotion-cause pair extraction. It alleviates problems such as label imbalance and has good results on the publicly available dataset.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and relates to an emotion - cause pair extraction based on knowledge - driven multi - classification and graph position embedding. Background Art

[0002] In recent years, emotion analysis has become a popular direction in natural language processing. Emotion cause extraction (ECE) is a branch of emotion analysis. Given a document and an emotion clause therein, the cause clause that gives rise to the emotion is to be found. Although emotion cause extraction has attracted extensive attention in natural language processing tasks, it requires that the emotion clauses must be pre - marked, which is difficult to carry out in actual scenarios. To solve this problem, a new task called emotion - cause pair extraction (ECPE) has been proposed. Different from the previous task of extracting emotion cause clauses, the emotion - cause pair extraction task regards the emotion clause and its corresponding cause clause as a whole, called an emotion - cause pair, and extracts both jointly. The extracted emotion - cause pairs contain rich emotion cause information and play a key role in in - depth emotion analysis. This task has attracted extensive attention in recent research.

[0003] Figure 1 is a document for the ECPE task and the extracted emotion - cause pairs. The document contains five consecutive clauses. C1 is an emotion clause with the emotion word "Excited", and at the same time C1 is also its own corresponding cause clause. The emotion "Excited" is caused by the phrase "See the paintings". Similarly, C4 is also an emotion clause with the emotion word "Guilty", but the cause that gives rise to the "Guilty" emotion is not included in C4. By reading other clauses in the document, it can be found that C5 is the corresponding cause clause of C4. The goal of this task is to extract the set of emotion - cause pairs: [(C1, C1), (C4, C5)].

[0004] Compared with the task of extracting the cause of a given emotion, emotion - cause pair extraction is a more challenging task. Previous studies on emotion - cause pair extraction mainly fall into two categories: pipeline models and end - to - end models. The pipeline model consists of two parts: in the first step, algorithms are designed to extract emotion clauses and cause clauses respectively, and in the second step, the emotion clauses and cause clauses extracted in the first step are matched. However, the pipeline model has serious defects. Since the errors generated in the first step may spread and affect the performance of the second step, the performance of the pipeline model is limited. To solve this problem, end - to - end emotion - cause pair algorithms have been proposed, and most existing end - to - end models use two - dimensional representations to represent emotion - cause pairs and make predictions based on different neural models.

[0005] Although these studies have improved the performance of emotion - cause pair extraction, there are still the following three problems. (1) Most relevant studies classify clauses into two types: emotion and cause, ignoring the fact that there are four types of clauses: emotion, cause, both emotion and cause, and neither emotion nor cause. The traditional binary classification used to detect emotion and cause may be misled by the other two types. (2) Emotion clauses and cause clauses are usually determined by some specific words, which make the clauses become emotion clauses or cause clauses. These unique words usually have specific properties known in advance, yet this prior knowledge is not included in recent work. In fact, human judgment of emotion - cause is also influenced by prior knowledge. (3) In existing work, the relative positions of true emotion - cause pairs are usually encoded or a position matrix is generated, which may be negatively affected by inaccurate emotion - hypothesis clauses and unbalanced document lengths. Summary of the Invention

[0006] The object of the present invention is to provide an emotion - cause recognition algorithm to overcome the shortcoming of incomplete distinction of clause types in existing emotion - cause models. The present invention takes emotion - cause pair extraction as a new knowledge - based multi - classification task and proposes a graph - based position embedding constraint. The present invention classifies clauses into four types: emotion clauses, cause clauses, emotion - cause clauses, and non - emotion - cause clauses. First, a sub - task is designed to extract these four types of clauses, which utilizes semantic embedding and prior emotion information. Then, a weighted directed graph is constructed for the positions of emotion - cause pairs in the training data set, and edge weight information is propagated to obtain the context information of the edges. Subsequently, by integrating in - degree, out - degree, and edge context information, position embedding is obtained. Finally, an emotion - cause pair extraction model based on window size is designed.

[0007] The technical solution of the present invention is as follows:

[0008] An emotion - cause pair extraction method based on knowledge - driven multi - classification and graph position embedding, comprising the following steps:

[0009] S1 Calculate the emotion score of the clause: It is generally considered that the emotion tendency of a clause containing emotion should be greater than that of a clause without emotion, and the emotion tendency of a clause containing a cause is usually also greater than that of a clause without any emotion - cause. Based on this, the emotion tendency of the clause is designed to be calculated. An emotion dictionary is imported, and each word in the emotion dictionary is assigned an emotion score; each document is read one by one, the word segmentation of each clause in the document is extracted, and it is matched with the words in the emotion dictionary and the corresponding emotion scores are assigned. If the word does not exist in the emotion dictionary, the score is recorded as zero, and then the emotion scores of the clauses are added up and stored;

[0010] S2 Document Semantic Embedding Learning: Replace the word segmentation in the clause with word vectors, and generate a word embedding matrix for each clause; Use a lexical-level bidirectional long short-term memory network to learn the context information of the words between clauses; Then add an attention mechanism to make the algorithm focus on the keywords in the sentence, and generate a sentence vector for each clause;

[0011] Input the obtained sentence vectors into a sentence-level bidirectional long short-term memory network of a specific semantic type to learn the context and generate a semantically specific sentence vector; The semantically specific sentence vectors include emotion-specific sentence vectors, reason-specific sentence vectors, emotion-reason-specific sentence vectors, and non-emotion-reason-specific sentence vectors;

[0012] S3 Joint Learning Classification for Clause Emotion Reason Judgment: Use four linear classifiers to distinguish different types of semantically specific clause vectors in step S2. Among them, when predicting, the emotion scores obtained in step S1 are also added to each classifier for judgment;

[0013] S4 Location Embedding Based on Graph Information:

[0014] There are two different emotion-reason pair location extraction strategies in the existing work: 1) Assume a sentence in the document as an emotion clause, and encode the relative positions of other clauses centered on this clause. For example, assume that the third sentence in the document is the emotion clause, then the relative position of the third sentence is 0, the relative position of the second sentence in this document is -1, and the relative position of the fourth sentence in the document is 1, and so on. However, there are obvious defects in this relative position editing algorithm. The algorithm assumes each sentence in the document as an emotion clause, which is of no help in extracting emotion clauses. 2) The real matrix generation algorithm of labels. The disadvantage of this method is that due to the different lengths of documents, problems such as weight imbalance will occur, and the obtained real label matrix is not universal, which has an incorrect impact on the algorithm. To alleviate the problem of weight imbalance, location embedding based on graph information is proposed. The label position information, the in-degree and out-degree of the graph are used as vectors to spread information between neighbors and generate location embedding. It is considered that if the number of occurrences of the emotion-reason pair at this position is higher, then the probability of this position as an emotion reason is higher. The in-degree and out-degree of the graph represent the probabilities of the cause clause and the emotion clause appearing at this position. The module consists of two parts: graph construction and generation of location embedding. The specific implementation is as follows:

[0015] S4.1 Graph Construction: Calculate the maximum value m of the number of clauses in all documents in the dataset, construct an unconnected graph without edges containing m nodes, establish edges according to the emotion reason clause types judged in S3 according to the positions of the emotion-reason pairs, and assign weights to the edges according to the number of occurrences; Initially, no edges are added.

[0016] S4.2 Graph Information Propagation and Location Generation: Use the GCN algorithm to learn the neighbor information of edges, generate edge sentence embeddings with context, and incorporate the in-degree and out-degree information of the graph to learn the context and generate the final location information; the in-degree and out-degree of the graph represent the probabilities of the appearance of the reason clause and the emotion clause at this location.

[0017] S5 Window-based Emotion-Reason Pair Extraction: Consider each sentence in the document as a candidate clause, and splice it with the clauses in the document to form emotion-reason candidate pairs; set the search window size, and for a candidate clause, only select the sentences within the search window range to generate emotion-reason candidate pairs, and splice the location information and combine with the emotion-reason pair probability judged in S4.2.

[0018] Furthermore, the emotion score of clause i in S1 where is the emotion score of the j-th word segment in clause i, and k is the number of word segments in clause i.

[0019] Furthermore, in S3, first perform clause emotion and reason classification, then perform clause emotion-reason classification, and finally perform clause non-emotion-reason classification;

[0020] The method for clause emotion-reason classification is: Splice the emotion scores with the emotion-specific sentence vector and the reason-specific sentence vector respectively, and send them into a linear classifier to obtain the probability of clause emotion classification and the probability of clause reason classification;

[0021]

[0022]

[0023] where is the emotion-specific sentence vector, is the reason-specific sentence vector, W e is the emotion trainable parameter, W c is the reason trainable parameter, b e is the emotion bias vector, b c is the reason bias vector, and are the distributions of the predicted emotion clause and reason clause respectively, s i is the emotion score of the clause, and i represents the i-th clause in this document;

[0024] The clause emotion-reason classification: When the probabilities of the emotion and reason of the clause are higher, theoretically the classification of the emotion-reason of the clause should be higher. Combine the clause feature vector with the clause emotion classification probability, clause reason classification probability, and emotion score to jointly judge the clause emotion-reason classification;

[0025]

[0026] Among them, is the distribution of the predicted emotional cause clauses, is the vector of the emotional cause characteristic sentence, W ec is the trainable parameter of the emotional cause, b ec is the bias vector of the emotional cause;

[0027] For the non-emotional cause classification of the clause: On the contrary to the emotional cause classification, when the probabilities of the clause emotional classification and the cause classification are lower, the non-emotional cause classification of the clause should be higher. The clause feature vector is concatenated with the probability of the clause non-emotional classification, the probability of the clause non-cause classification, and the clause emotion score to jointly judge the non-emotional cause classification of the clause;

[0028]

[0029] Among them, is the distribution of the predicted non-emotional cause clauses, is the vector of the non-emotional cause characteristic sentence, W n is the trainable parameter of the non-emotional cause, b n is the bias vector of the non-emotional cause; and are the probabilities of the non-emotional classification and the non-cause classification respectively, where

[0030] The loss of the joint judgment of the clause emotional cause is as shown in the formula:

[0031]

[0032] where j is different clause classification types, y represents the predicted label, represents the true label.

[0033] Furthermore, in S4.1, when the document contains a set of emotion-cause pairs P ij , where i represents the emotional clause and j represents the cause clause; if there is no cause pair between the i node and the j node in the graph, add a new edge E ij = <i, j>, and set the value of this edge to 1; if this emotion-cause pair P ij appears n times, then set the value of this edge to n.

[0034] Furthermore, in S4.2, multiple rounds of GRU are added to obtain the neighbor information of the edges in the graph, and the result of the last round of iteration is used as the result p ij of the edge position embedding, where the result of the (t + 1)-th round is derived from the result of the t-th round:

[0035] P(t + 1) = GRU(concat(P(t), P(0)))

[0036] Among them, P(0) is the initial value of the edge.

[0037] Use BiLSTM to learn the in-degree and out-degree of the edge and the edge position embedding, and generate the position embedding representing the position information. The formula is as follows:

[0038]

[0039] Among them, p i,j is the edge position embedding after iteration, is the in-degree of the i-th point in the graph, representing the probability of this position as a cause, is the out-degree of the j-th point in the graph, representing the probability of this position as an emotion.

[0040] Furthermore, a fully connected layer is used in S5 to predict the final emotion-cause pair. The formula is as follows:

[0041]

[0042] Among them W is the weight parameter, b is the bias, i is the i-th node in the document as the candidate emotion clause, and j is the j-th node in the document as the candidate cause clause;

[0043] y ij is the emotion-cause pair prediction label, is the true label of the emotion-cause pair. The Loss calculation formula is as follows:

[0044]

[0045] Among them, d is the number of emotion-cause pairs in the document. i and j indicate the search range of the emotion-cause pair. According to the Chinese language habit, the cause clause that usually causes emotion is near the emotion clause. Preferably, j = [i - 2, i + 2].

[0046] Furthermore, the total training Loss formula of the algorithm is as follows: L all = L sub + λ1L pair + λ2||Θ||;

[0047] Among them, L sub is the loss of the combined judgment of the clause emotion and cause, L pair is the loss of the emotion-cause pair judgment, λ1 ∈ (0, 1) is the weight, and λ2||Θ|| is the hyperparameter in the model.

[0048] The present invention proposes an emotion-cause pair extraction system based on knowledge-driven multi-classification. As Figure 2 shown, it includes:

[0049] Emotion Score Calculation Module: For each clause in the document, query the emotion score of each word, sum and calculate the clause emotion score and save it. The formula is as follows:

[0050]

[0051] Semantic Learning Module: Used to learn the context of adjacent words and adjacent sentences from the clause, add an attention mechanism, and generate a clause embedding. It includes emotion-specific clause embeddings, reason-specific clause embeddings, emotion-reason-specific clause embeddings, and non-emotion-reason-specific clause embeddings for subsequent classification.

[0052] Emotion-Reason Classification Clause Classification Module: Used to calculate the probability that a sentence belongs to different types of clauses, and the types include emotion, reason, emotion-reason, and non-emotion-reason; the emotion distribution and reason distribution are predicted from the clause sentiment score and semantic-specific clause embedding; the emotion-reason prediction is generated from the distributions of emotion and reason and the emotion-reason-specific clause embedding; the distribution of non-emotion-reason is generated from the non-emotion, non-reason, and non-emotion-reason-specific clause embeddings.

[0053] Graph Information Embedding Module: Used to generate location information, which is divided into two steps: graph construction and graph information propagation and location generation.

[0054] Graph Construction: Generate an acyclic unconnected graph with the maximum number of nodes, establish edges according to the positions of emotion-reason pairs, and assign weights to the edges according to the number of occurrences.

[0055] Graph Information Propagation and Location Generation: Use the GCN algorithm to learn the neighbor information of the edges, generate edge clause embeddings with context, and add the in-degree and out-degree information of the graph to learn the context to generate the final location information.

[0056] Window-Based Emotion-Reason Pair Extraction Module: Used to obtain the final emotion-reason pairs, and improve the accuracy by restricting the search window size. The window is set to 2, and only the clauses at the positions of the two sentences before and after the candidate emotion clause are taken as the candidate reason clause range each time.

[0057] The beneficial effects of the present invention are as follows: The emotion-reason pair extraction algorithm of the present invention is driven by external knowledge, and improves the accuracy of emotion-reason pairs through a series of algorithms. It alleviates problems such as label imbalance and avoids the disadvantages of binary classification.

[0058] Compared with the existing emotion-reason pairs, the present invention has the following characteristics:

[0059] (1) The beneficial effects of the present invention classify the document clauses into four categories, and design a knowledge-based multi-classification sub-task, which mines higher-level semantic information through prior knowledge and improves the accuracy of the algorithm.

[0060] (2) Integrate the propagated edge context information, the in-degree of the node, and the out-degree of the node into the BiLSTM to make full and more accurate use of the position information of the emotion cause pairs.

[0061] (3) The effectiveness of the model was tested on the publicly available ECPE dataset, and the superiority of the proposed model compared with many competing baselines was demonstrated. Description of the Drawings

[0062] Figure 1 is a document of the ECPE task and the extracted emotion cause pairs

[0063] Figure 2 is the flowchart of the knowledge-driven emotion-cause pair extraction algorithm of the present invention.

[0064] Figure 3 is the schematic diagram of the module structure of the knowledge-driven emotion-cause pair extraction algorithm of the present invention.

[0065] Figure 4 is the schematic diagram of the calculation of the emotion score of each sentence in the document of the embodiment of the present invention. Detailed Embodiment

[0066] The following is an explanation of the present invention in combination with the drawings and specific embodiments:

[0067] As Figure 2 shown, a method for extracting emotion-cause pairs based on knowledge-driven multi-classification and graph position embedding includes the following steps:

[0068] A. Calculation of the clause emotion score: Import the Boson emotion dictionary and read each document one by one. Segment each clause in the document and calculate the emotion score of each word. If the word does not exist in the emotion dictionary, the score is recorded as zero, and the clause emotion score is calculated by adding and stored.

[0069] Among them, the Boson emotion dictionary is a Chinese emotion dictionary, which gives each word an emotion score. In the dictionary, according to the strength of the emotion expressed by the word, the scores are from high to low. And positive or negative emotion scores are assigned according to the emotion polarity of the word.

[0070] B. Document semantic embedding learning: It includes word-level embedding learning and sentence-level embedding learning of sentences.

[0071] The method for lexical-level embedding learning is as follows: First, convert words into word vectors. Each word in the clause corresponds to a word embedding with a dimension of 200. Arrange the word embeddings in the original sentence order to generate a clause word embedding matrix. Use a lexical-level bidirectional long short-term memory network (Bi-LSTM) to learn the context information of words between clauses. Then add an attention mechanism to make the algorithm focus on the key words in the sentence, and finally generate a sentence embedding for each sentence.

[0072] The method for sentence-level embedding learning is as follows: Send the sentence embeddings obtained above into a sentence-level Bi-LSTM to learn the context and generate semantically specific sentence embeddings.

[0073] Among them, the semantically specific sentence embedding refers to the use of four Bi-LSTM models. Although the network structures are the same, different weight parameters are randomly initialized to calculate emotion-specific, reason-specific, emotion-reason-specific, and non-emotion-reason-specific sentence vectors for subsequent joint classification of clauses.

[0074] C. Joint learning classification for clause emotion-reason judgment: First, perform clause emotion and reason classification, then perform clause emotion-reason classification, and finally perform clause non-emotion-reason classification.

[0075] The method for clause emotion-reason classification is as follows: Concatenate the emotion score of the emotion-specific sentence vector and the reason-specific sentence vector and send them into a linear classifier to obtain the probability of clause emotion classification and the probability of clause reason classification.

[0076] Clause emotion-reason classification: The higher the probabilities of the emotion and reason of the clause, the higher the classification of the clause's emotion-reason should be theoretically. Concatenate the clause feature vector with the emotion classification probability, reason classification probability, and emotion score to jointly judge the clause emotion-reason classification.

[0077] Clause non-emotion-reason classification: Contrary to the emotion-reason classification, when the probabilities of clause emotion classification and reason classification are lower, the clause non-emotion-reason classification should be higher. Concatenate the clause feature vector with the probability of non-emotion classification, the probability of non-reason classification, and the clause emotion score to jointly judge the clause non-emotion-reason classification.

[0078] D. Position embedding based on graph information: Calculate the maximum value m of the number of clauses in all documents in the dataset, construct a graph containing m nodes, traverse the emotion-reason pair positions in all documents, and create edges of the graph according to the emotion-reason pair positions. For duplicate edges, each time a duplicate position is read, the edge value is incremented by one. Use the GRU algorithm to learn the edge neighbor information, and use the LSTM to jointly learn the edge and the in-degree and out-degree neighbor information of the graph.

[0079] E. Window-based Emotion-Cause Pair Extraction: Each sentence in the document is used as a candidate clause, which is concatenated with the clauses in the document to form emotion-cause candidate pairs. Set the search window size. For a candidate clause, only the sentences within the search window range are selected to generate emotion-cause candidate pairs, and the concatenated position information is jointly used to judge the probability of the emotion-cause pair.

[0080] Embodiment

[0081] Suppose an input document contains three sentences: "At that time, I bought a new house in Luohe. Immersed in the joy of becoming a citizen of Luohe. Feeling finally able to breathe a sigh of relief", where the window size is set to 2.

[0082] (1) Calculation of Clause Emotion Score

[0083] Tokenize each clause in the document and remove punctuation marks, splitting the clause into sentences containing only words, as Figure 4 shown. Calculate the emotion score of each sentence in the document. First, read the emotion dictionary, which is stored in the form of <key, value>. Store the emotion words as keys and the emotion scores as values for convenient query. Initially, the emotion score of each sentence is zero. Query whether each word in the clause exists in the emotion dictionary. If it exists in the emotion dictionary, obtain the emotion score and add it to the clause emotion score. As shown in the formula:

[0084]

[0085] where S i represents the emotion score of the i-th sentence in the document. Suppose there are k words in the sentence. As can be seen from the figure, C2 is the emotion clause in the document, and its emotion score is also the highest in this document, which is 3.22. The score of C1 is slightly higher than that of C3. From the sentence meaning, it can also be seen that C1 is the cause clause that causes the emotion of C2.

[0086] (2) Document Semantic Embedding Learning

[0087] For each sentence in the document, after tokenization, replace it with word vectors. Each word in the clause corresponds to a word embedding with a dimension of 200. Arrange the word embeddings to generate the clause word embedding matrix C i ∈R k*200 , where k represents the number of words in the sentence. Among them, C i =(w i1 , w i2 , …, w ik ), and there are k words in the sentence. First, use the lexical-level bidirectional long short-term memory artificial neural network (Bi-LSTM) algorithm to learn the context information at the clause level in the document, as shown in the formula:

[0088]

[0089] Considering that the extraction of emotional clauses and causal clauses is usually affected by certain words or phrases, an attention mechanism is added to this method for learning to generate new embeddings that combine self-attention, as shown in the formula:

[0090]

[0091] Among them, W1 and W2 are trainable parameters. To learn the features of different types of sentences, four sentence-level Bi-LSTMs are introduced to learn the context information between clauses. It should be noted that although the same network structure is used, this method uses different weights to generate four semantics-specific sentence vectors: emotional sentence vector, causal sentence vector, emotional-causal sentence vector, and non-emotional-causal sentence vector. The emotional-semantics-specific sentence vector is shown in the formula:

[0092]

[0093] The acquisition methods of other semantics-specific sentence embeddings are similar to those of the emotional-semantics-specific sentence embeddings. The cause-semantics-specific sentence embedding The emotional-cause-specific sentence embedding And the non-emotional-cause-specific sentence embedding

[0094] (3) Joint learning classification for clause emotional cause judgment

[0095] Based on the above analysis, four linear classifiers are added to this method for the classification and extraction of different types of clauses. To obtain better results, semantics-specific sentence embeddings and Bosen emotional scores are used to predict emotional and causal clauses. The formula is as follows:

[0096]

[0097]

[0098] Among them, W e Is the trainable parameter for emotion, W c Is the trainable parameter for cause, b e Is the emotion bias vector, b c Is the cause bias vector, y e And y c Are the distributions of the predicted emotional clause and causal clause, s iis the sentiment score of the fraction obtained from the previous text. Here, i represents the i-th clause in this document, and the same applies hereinafter. It should be noted that if a clause has a high score in the separate classifications of sentiment and reason, then theoretically the probability of it in the sentiment-reason classification should increase. Therefore, in the prediction of other clause classifications, this method combines the probabilities of sentiment and reason for joint judgment. As shown in the formula:

[0099]

[0100] The extraction of non-sentiment reasons is similar to that of sentiment reasons. The difference is that it is observed that the lower the probabilities of sentiment and reason extraction, the higher the probability of non-sentiment reasons. Therefore, this method adds the probabilities of non-sentiment classification and non-reason classification, denoted as and where The calculation of non-sentiment reasons is as shown in the formula:

[0101]

[0102] The loss of the joint judgment of clause sentiment and reason is as shown in the formula:

[0103]

[0104] where j is different clause classification types, y represents the predicted label, represents the true label.

[0105] (4) The position embedding based on graph information consists of two parts: graph construction and generating position embedding. The specific implementation is as follows:

[0106] Graph construction: Calculate the maximum value m of the number of clauses in all documents in the dataset, and construct a graph with m nodes and no edges. Read all the sentiment-reason pair positions in the dataset. For example, a document contains a set of sentiment-reason pairs P ij , where i represents the sentiment clause and j represents the reason clause. If there is no reason pair between node i and node j in the graph, add a new edge E ij = <i, j>, and set the value of this edge to 1. If this sentiment-reason pair P ij appears n times, then set the value of this edge to n.

[0107] Graph information propagation and position embedding generation: Use multiple rounds of GRU to obtain the neighbor information of the edges in the graph. Take the result of the last round of iteration as the result p ij of the position embedding. The result of the (t + 1)-th round is derived from the result of the t-th round:

[0108] P(t + 1) = GRU(concat(P(t), P(0)))

[0109] Use BiLSTM to learn the in-degree, out-degree, and positional embeddings of edges, and generate positional embeddings representing positional information. The formula is as follows:

[0110]

[0111] (5) Window-based emotion-cause pair extraction

[0112] Table 1

[0113]

[0114]

[0115] Sentence vectors using emotion features Sentence vectors of cause features Sentence vectors of emotion-cause features Emotion clause prediction score Cause clause prediction score Emotion-cause clause prediction score and positional embeddings Jointly extract emotion-cause pairs. Specifically, it is formulated as Finally, use a fully connected layer to predict the final result. The formula is as follows:

[0116]

[0117] Replaced the calculation formula of the emotion-cause pair when i = j, and adopted separate extraction of emotion-cause clauses. y ij is the emotion-cause pair prediction label, is the true label of the emotion-cause pair. The Loss calculation formula is as follows:

[0118]

[0119] where d is the number of emotion-cause pairs in the document. i and j indicate the search range of the emotion-cause pair. As can be seen from Table 1, the cause clause that usually causes emotion is near the emotion clause. Among them, 95.43% of the cause clauses are within two sentences before and after the emotion clause, and 85.5% of the cause clauses are within one sentence before and after the emotion clause. Therefore, set a window to limit the search range and improve the efficiency of the algorithm, where j = [i - 2, i + 2]. The total training Loss formula of the algorithm is as follows:

[0120] L all = L sub + λ1L pair + λ2||Θ||.

Claims

1. A method for extracting emotional reasons based on knowledge-driven multi-classification, characterized in that, It includes the following steps: S1 Calculate the clause emotion score: Import an emotion dictionary, and assign an emotion score to each word in the emotion dictionary; Read each document one by one, extract the word segmentation of each clause in the document, match it with the words in the emotion dictionary and assign the corresponding emotion score. If the word does not exist in the emotion dictionary, the score is recorded as zero, and then add them up to calculate the clause emotion score and store it; S2 Document semantic embedding learning: Replace the word segmentation in the clause with word vectors, and generate a word embedding matrix for each clause; Use a word-level bidirectional long short-term memory network to learn the context information of the words between clauses; Then add an attention mechanism to generate a sentence vector for each clause; Input the obtained sentence vectors into a sentence-level bidirectional long short-term memory network of a specific semantic type to learn the context and generate a semantically specific sentence vector; The semantically specific sentence vector includes an emotion-specific sentence vector, a reason-specific sentence vector, an emotion-reason-specific sentence vector, and a non-emotion-reason-specific sentence vector; S3 Joint learning classification for clause emotion reason judgment: Use four linear classifiers to distinguish different types of semantically specific clause vectors in step S2. Among them, when predicting, the emotion score obtained in step S1 is also added to each classifier for judgment; S4 Position embedding based on graph information: S4.1 Graph construction: Calculate the maximum value m of the number of clauses in all documents in the dataset, construct an unconnected graph without edges containing m nodes, and establish edges according to the position of the emotion-reason pair according to the type of emotion reason clause judged in S3, and assign weights to the edges according to the number of occurrences; S4.2 Graph information propagation and position generation: Use the GCN algorithm to learn the neighbor information of the edges, generate edge sentence embeddings containing context, and add the in-degree and out-degree information of the graph to learn the context to generate the final position information; The in-degree and out-degree of the graph represent the probabilities of the occurrence of reason clauses and emotion clauses at this position; S5 Emotion-reason pair extraction based on window: Use each sentence in the document as a candidate clause, and splice it with the clauses in the document to form an emotion-reason candidate pair; Set the search window size. For a candidate clause, only select the sentences within the search window range to generate an emotion-reason candidate pair, and splice the position information and the emotion-reason pair probability judged in S4.

2.

2. The method for extracting emotional cause pairs based on knowledge-driven multi-classification according to claim 1, wherein, The emotional score of clause i in S1 wherein is the emotional score of the j-th participle in clause i, and k is the number of participles in clause i.

3. The method for extracting emotional cause pairs based on knowledge-driven multi-classification according to claim 1, characterized in that In S3, first perform clause emotion and reason classification, then perform clause emotion reason classification, and finally perform clause non-emotion reason classification; The method for clause emotion reason classification is as follows: Splice the emotion score with the emotion-specific sentence vector and the reason-specific sentence vector respectively, and send them into a linear classifier to obtain the probability of clause emotion classification and the probability of clause reason classification; Among them, is the sentence vector of the emotion feature, is the sentence vector of the cause feature, W e is the trainable parameter of emotion, W c is the trainable parameter of cause, b e is the emotion bias vector, b c is the cause bias vector, and are the distributions of the predicted emotion clause and cause clause respectively, s i is the emotion score of the clause, where i represents the i-th clause in this document; The clause emotion reason classification: Jointly judge the clause emotion reason classification by splicing the clause feature vector with the clause emotion classification probability, the clause reason classification probability, and the emotion score; Among them, is the distribution of the predicted emotional reason clause, is the emotional reason feature sentence vector, W ec is the trainable parameter of the emotional reason, b ec is the emotional reason bias vector; The clause non-emotion reason classification: Jointly judge the clause non-emotion reason classification by splicing the clause feature vector with the probability of clause non-emotion classification, the probability of clause non-reason classification, and the clause emotion score; Among them, is the distribution of predicted non-emotional reason clauses, is the non-emotional reason feature sentence vector, W n is the non-emotional reason trainable parameter, b n is the non-emotional reason bias vector; and are the probabilities of non-emotional classification and non-reason classification respectively, where The loss of joint judgment of clause emotion reason is as shown in the formula: where j is different clause classification types, y represents the predicted label, representing the true label.

4. The method for extracting emotional cause pairs based on knowledge-driven multi-classification according to claim 1, characterized in that S4.1 When the document contains a set of emotion - reason pairs P ij , where i represents the emotion clause and j represents the reason clause; if there is no reason pair between the i - node and the j - node in the graph, add a new edge E ij = <i, j>, and set the value of this edge to 1; if this emotion - reason pair P ij appears n times, then set the value of this edge to n.

5. The method for extracting emotional cause pairs based on knowledge-driven multi-classification according to claim 1, characterized in that In S4.2, multiple rounds of GRU are added to obtain the neighbor information of the edges in the graph, and the result of the last round of iteration is used as the result pij of the edge position embedding. The result of the (t + 1)-th round is derived from the result of the t-th round as follows: P(t + 1) = GRU(concat(P(t), P(0))) where P(0) is the initial value of the edge; BiLSTM is used to learn the in-degree and out-degree of the edge and the edge position embedding, and generate the position embedding representing the position information. The formula is as follows: where pij is the edge position embedding after iteration, is the in-degree of the i-th point in the graph, representing the probability of this position as a cause, is the out-degree of the j-th point in the graph, representing the probability of this position as an emotion.

6. The method for extracting emotional cause pairs based on knowledge-driven multi-classification according to claim 3, characterized in that In S5, a fully connected layer is used to predict the final emotion-cause pair. The formula is as follows: where W is a weight parameter, b is a bias, i is the i-th node in the document as a candidate sentiment clause, and j is the j-th node in the document as a candidate reason clause; y ij is the predicted label of the emotion - cause pair, is the true label of the emotion - cause pair. The Loss calculation formula is as follows: where d is the number of emotion-cause pairs in the document, and i and j indicate the search range of the emotion-cause pair.

7. The method for extracting emotional cause pairs based on knowledge-driven multi-classification according to claim 6, wherein In S5, j = [i - 2, i + 2].

8. The method for extracting emotional cause pairs based on knowledge-driven multi-classification according to claim 6, wherein The total training Loss formula of the algorithm is as follows: L all = L sub + λ1L pair + λ2||Θ||; Among them, L sub is the loss for the combined judgment of the clause emotion cause, and L pair is the loss for the emotion-cause pair judgment. λ1 ∈ (0, 1) is the weight, and λ2||Θ|| is the hyperparameter in the model.

9. An emotion cause pair extraction system based on knowledge-driven multi-classification using the method according to any one of claims 1-8, characterized in that, It includes the following modules: Emotion score calculation module: used to query the emotion score of each word for each clause in the document, add them up to calculate the clause emotion score and save it; Semantic learning module: used to learn the context of adjacent words and adjacent sentences from the clause, add an attention mechanism, and generate the clause embedding; the clause embedding includes emotion-specific clause embedding, cause-specific clause embedding, emotion-cause-specific clause embedding, and non-emotion-cause-specific clause embedding for subsequent classification; Emotion-cause classification clause classification module: used to calculate the probability that a sentence belongs to different types of clauses, and the types include emotion, cause, emotion-cause, and non-emotion-cause; the emotion distribution and cause distribution are predicted from the clause sentiment score and the semantic-specific clause embedding; the emotion-cause prediction is generated from the distributions of emotion and cause and the emotion-cause-specific clause embedding; the distribution of non-emotion-cause is generated from the non-emotion, non-cause, and non-emotion-cause-specific clause embedding; Graph information embedding module: used to generate position information, including graph construction, graph information propagation, and position generation; The graph construction: generate an acyclic unconnected graph without edges with the maximum number of nodes, establish edges according to the positions of the emotion-cause pairs, and assign weights to the edges according to the occurrence times; The graph information propagation and position generation: use the GCN algorithm to learn the neighbor information of the edges, generate the edge clause embedding with context, and add the in-degree and out-degree information of the graph to learn the context to generate the final position information.

10. The system according to claim 9, characterized in that, It also includes a window-based emotion-cause pair extraction module: used to obtain the final emotion-cause pair. By restricting the search window size, the window is set to 2, and only the clauses at the positions of the two sentences before and after the candidate emotion clause are taken as the candidate cause clause range each time.

Images