Method for Interactive Attribute Term and Sentiment Association Extraction Model

Abstract

The present invention relates to the technical field of emotional extraction models for text content, and particularly to a method for an interactive attribute term and emotion joint extraction model. Aiming at the problems that existing applications are difficult to effectively learn the features of attribute terms and the research content of attribute term and emotion joint extraction is lacking, the following technical solutions are proposed: It includes an attribute term extraction module, an emotion classification module, and an interactive sharing unit, and comprises the following steps: Step 1: Obtain the emotional tendency of words in a sentence; Step 2: The interactive sharing unit enables the two modules to learn from each other; Step 3: Combine the task labels of the attribute term extraction task and the emotion classification task. The present invention makes full use of external emotional resources, enhances emotional analysis through two-stage processing, improves the accuracy of emotion extraction in sentences, and the interactive sharing unit improves the learning effect of the module, which is helpful for the result prediction of feature vectors, and is mainly applied to the joint extraction of attribute terms and text emotions.

Description

Technical Field

[0001] The present invention relates to the technical field of emotional extraction models for text content, and particularly to a method for an interactive attribute term and emotion joint extraction model. Background Art

[0002] With the development of communication technology and the popularization of mobile terminal devices, the number of Internet users is increasing continuously. People express their views on hot events on the Internet, share their evaluations of Internet celebrity foods, popular commodities, and tourist attractions. Users communicate fully here, express their views and attitudes, and convey their experiences and feelings to other users. Making full use of these text information is helpful for public opinion analysis and improving product quality, which is of great significance to both the industrial and academic fields. However, it is obviously inefficient and difficult to process such a large amount of text data manually. It is necessary to rely on the efficient processing of computers and artificial intelligence algorithms for filtering and analysis, thus giving rise to the research topic of text sentiment analysis.

[0003] However, most of the existing research is carried out for ATE and ASC, which limits the application of emotion extraction in real life. There are difficulties in the existing sentence emotion extraction methods, such as less training data and too many irrelevant words in the data, which make it difficult for the model to effectively learn the features of attribute terms. Moreover, the research content on the joint extraction of attribute terms and emotions in the existing research is relatively scarce. In view of this, we propose a method for an interactive attribute term and emotion joint extraction model. Summary of the Invention

[0004] The purpose of the present invention is to propose a method for an interactive attribute term and emotion joint extraction model for the problems in the background art that the existing applications are difficult to effectively learn the features of attribute terms and the research content on the joint extraction of attribute terms and emotions is relatively scarce.

[0005] The technical solution of the present invention: A method for an interactive attribute term and emotion joint extraction model, including an attribute term extraction module, an emotion classification module, and an interactive sharing unit, specifically including the following generation steps:

[0006] Step 1: A neural network combined with external emotion resources obtains the emotion tendency of each word in the sentence;

[0007] Step 2: The attribute term extraction module is combined with the emotion classification module, and an interactive sharing unit is used to make the feature representations of the attribute term extraction module and the emotion classification module learn from each other;

[0008] Step 2.1: Obtain the feature vector matrix of the input text and three emotion resources, enhance the corresponding features in the input text in two stages using emotion resources, and predict the results of the enhanced text feature vectors;

[0009] Step 2.2: Combine the attribute terms and the sentiment tendency of each word obtained from the DP-GCN model and the Two-stage MSE model, and propose an interactive sharing unit ISU. The interactive sharing unit ISU is used to jointly learn two independent subtasks;

[0010] Step 3: Combine the obtained attribute term extraction task labels and sentiment classification task labels to obtain the final output result, that is, the attribute term - sentiment label pair, and obtain the final sentence prediction result.

[0011] Preferably, the attribute term extraction module is an attribute term extraction model based on dependency analysis and GCN. The attribute term extraction model uses a combination of a general domain vector and a specific domain vector as the vector representation of each word, and thus proposes an adjacency matrix construction method based on dependency analysis and PMI. The semantic features obtained through the LSTM network and the sentence structure features obtained through the GCN network are combined by weights to obtain the final vector representation of the input text.

[0012] Preferably, the three sentiments in Step 2.1 are positive (POS), negative (NEG), neutral (NEU), and irrelevant word (0). The word vector is represented by the BERT model, the sentence is represented by S, and the vector matrix of the sentence based on BERT is The vector matrix of sentiment words The vector matrix of intensity words The vector matrix of negation words where t, m, k, p respectively represent the length of the input text and the number of sentiment words, intensity words, and negation words, and d represents the dimension of the word vector.

[0013] Preferably, the first-stage enhancement of the sentiment resources in the two-stage in Step 2.1 is used to establish the association matrix at the word level between the input text and the three sentiment resources. The enhancement in the first stage includes the following processing steps: Step 1: Obtain each association matrix through dot product operation; the calculation method formula is: M s =(w b ) T .w s ∈R t*m 、M i =(w b ) T .w i ∈R t*k 、M n =(w b ) T .w n ∈R t*p ; where: M sRepresents the association matrix between the input text and sentiment words. The elements in each row of the matrix represent the semantic similarity between each word in the input text and each sentiment word; M i and M n respectively represent the association matrix between the input text and degree words and the association matrix between the input text and negative words. The meaning of each element in the matrix is similar to that of the elements in M s ;

[0014] Step 2: For the sentiment word association matrix M s After obtaining it, calculate its row average vector and column average vector respectively through average pooling and column average vector The specific calculation method is as follows: where and are each row vector and each column vector in the sentiment word association matrix M s respectively. Similarly, find the row average vector and column average vector of the degree word association matrix M i and the negative word association matrix M n ;

[0015] Step 3: Combine the vector matrix W of the input text in Step 2 b and the column average vector of the sentiment word association matrix to calculate the input text X enhanced by sentiment words s , and calculate the sentiment word matrix combining the input text information through W b and the row average vector of the association matrix And perform similar operations on degree words and negative words in turn; the calculation method is as follows:

[0016] Step 4: The vector representation of the input text after being enhanced by three types of sentiment resources is the sum of the input text vector matrices related to the above three types of sentiment resources. The calculation method is as follows: X c =X s +X i +X n .

[0017] Preferably, in the second-stage enhancement of sentiment resources in Step 2.1, it is used to enhance the corresponding features of the hidden layer, specifically including the following steps: Step 1: After obtaining the input text feature matrix X c combining the information of three types of sentiment resources, input it into the LSTM network to obtain the hidden layer variable H c combining the context semantic information. The calculation method is as follows: H c =LSTM(X​c );

[0018] Step 2: Obtain the hidden layer H c Then, using the sentiment word matrix combined with the input text information and the degree word matrix combined with the input text information obtained in the first stage and the negation word matrix combined with the input text information enhance the corresponding features of the hidden layer matrix H of the input text again; The calculation method is as follows: c The calculation method is as follows: where represents each column vector in the sentiment word matrix, then q is obtained by performing a pooling operation on the sentiment matrix; s is obtained by performing a pooling operation on the sentiment matrix; represents the hidden layer vector of each word in the input text The weight score of the sentiment word direction of each word in the whole sentence, u s and W s are parameter matrices, tanh represents the activation function; α i is the weight of each word after normalization; is the final vector of the word after sentiment word enhancement, and the sentence after sentiment word enhancement is represented as O s ;

[0019] Step 3: According to O in Step 2 s Obtain the sentence representation O enhanced by degree words and the sentence representation O enhanced by negation words i and the sentence representation O enhanced by negation words n , the final vector representation of each sentence is the sum of the representations of the three sentences, and the calculation method is as follows: O = O s + O i + O n .

[0020] Preferably, the prediction of the result in Step 2.1 is reflected by the output module. After obtaining the final vector O of each word i , the feature vector of each word is converted into a label vector z = [p 1 , p 2 ,... p k through a fully connected layer. Each element p i in z represents that the word is a label c in the label set class = [c 1 , c 2 ,... c k , and the label c iThe possibility, k represents the number of labeled tags. The prediction result maps each value in the label vector to the probability of the corresponding label through the softmax function, and then obtains the index of the maximum value in the probability vector through the argmax function; in the experimental dataset, the weighted cross-entropy loss function with L2 regularization term is adopted for the sentiment classification task. The weighted cross-entropy loss function of the regularization term is used to solve the problem of unbalanced label quantities. The formula for the loss calculation method of the sentiment classification model is: where y c is the One-Hot vector of the true label of the sample, w c is the weight of each category, μ is the L2 regularization coefficient, and Θ is the set of parameters.

[0021] Preferably, in the corresponding feature enhancement of the input text in step 2.1 in two stages, it further includes the sentiment classification algorithm of Two-stage MSE, and the algorithm is:

[0022] Row1: Begin

[0023] Row2: Read each text sentence S in the data i

[0024] Row3: Set the number of iterations as N and the batch size as B;

[0025] Row4: For each S i in S

[0026] Row5: Obtain the vector representation W of the input sentence through the Bert model b

[0027] Row6: Obtain the vector representations W s 、W i 、W n

[0028] Row7: End for

[0029] Row8: Use the formula: M s =(w b ) T .w s ∈R t*m 、M i =(w b ) T .w i ∈R t*k 、M n =(w b ) T .w n ∈R t*p Calculate the sentiment word association matrix Ms Degree word association matrix M i Negative word association matrix M n

[0030] Row9: Use the formula to obtain the input text X with enhanced sentiment words s Input text X with enhanced degree words i Input text X with enhanced negative words n and sentiment words combined with input text information Degree words combined with input text information Negative words combined with input text information

[0031] Row10: Use the formula X c = X s + X i + X n to calculate the sentence representation X enhanced by three types of sentiment resources in the first stage c

[0032] Row11: Input X c into the LSTM, and use the formula H c = LSTM(X c ) to obtain the hidden layer vector representation

[0033] Row12: Use the formula to enhance the input text again respectively using

[0034] Row13: Use the formula O = O s + O i + O n to calculate the sentence representation O with enhanced sentiment words in the second stage

[0035] Row14: Obtain the sentiment polarity of each word in the sentence

[0036] Row15: For each n in N

[0037] Row16: For each b in B

[0038] Row17: Use the formula to calculate the error loss

[0039] Row18: Use the Adam gradient descent optimization algorithm to update the network parameters

[0040] Row19: End for ​

[0041] Row 20: End for

[0042] Row 21: End.

[0043] Preferably, the interactive sharing unit is used to calculate the correlation of the word feature representations in the tasks of two modules, and adjust the features of each word itself in combination with the correlation; specifically, it includes the following steps: Step 1: Calculate the correlation vector of each word in the attribute term extraction module and the sentiment classification module The calculation method formula is: where a and s respectively represent the attribute term extraction task and the sentiment classification task; represents the feature vector of word i obtained through the attribute term extraction task, represents the feature vector of word j obtained through the sentiment classification task, tanh is the activation function, and G is the parameter matrix; Step 2: After obtaining the correlation vector of word i and word j, transform it into a weight score through the formula ; where is the parameter vector, is the association score; the set of correlation vectors between words is represented by the matrix S a-s and S s-a ; Step 3: Enhance the feature matrices of the calculation attribute term extraction module and the sentiment classification module respectively through the correlation matrix; the calculation method formula is: H a = H a + softmax(S a -s )H s , H s = H s + softmax(S s-a )H a ; where H a and H s are the sentence representations enhanced by the interactive sharing unit, and are used to predict the label pair of the statement in Step 3 after being processed by the output module.

[0044] Preferably, the output module uses the DP-GCN model to output, and the sentiment classification model has two selection methods for multi-word attribute terms: The first case: When the sentiment tendencies of the words forming the same attribute term are inconsistent, the majority principle is used; The second case: When the sentiment tendencies are inconsistent and the occurrences of various sentiments are the same, the sentiment that appears first is selected.

[0045] Compared with the prior art, the present invention has the following beneficial technical effects:

[0046] 1. The present invention enhances the feature vectors of the input text by introducing external sentiment resources, combines the proposed DP-GCN model with the Two-stage MSE model, and enables the feature vectors in two independent modules, namely the attribute term extraction module and the sentiment classification module, to learn from each other through an interaction sharing unit, so as to obtain the final prediction result. By making full use of external sentiment resources, the sentiment judgment of sentences is improved. Moreover, through the Two-stage MSE model, the features of the input text are mutually enhanced, and the interaction sharing unit (ISU) that mutually enhances the features of the attribute term extraction module and the sentiment classification module increases the interaction between the two modules, which is beneficial to enhancing the sentiment features in the sentence and making the extraction of sentiment in the sentence more accurate.

[0047] 2. The present invention makes full use of external sentiment resources, adopts a two-stage sentiment analysis model with multi-sentiment resource enhancement to improve the accuracy of sentiment extraction in sentences, uses an interaction sharing unit to improve the learning effect of the module, helps to predict the result of the feature vector in the sentence, enriches text sentiment analysis, and is suitable for the analysis and use of a wide range of Internet text data. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a schematic diagram of the calculation process of the interaction sharing unit in the method of the interactive attribute term and sentiment joint extraction model;

[0049] Figure 2 is a histogram of the F1 value of the ablation experiment in this embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] The following further describes the technical solutions of the present invention in conjunction with the drawings and specific embodiments.

[0051] Embodiment

[0052] As Figure 1-2 shown, the method of the interactive attribute term and sentiment joint extraction model proposed by the present invention includes an attribute term extraction module, a sentiment classification module, and an interaction sharing unit, and specifically includes the following generation steps:

[0053] Step 1: A neural network combined with external sentiment resources obtains the sentiment tendency of each word in the sentence;

[0054] Step 2: The attribute term extraction module and the sentiment classification module are combined, and an interaction sharing unit is used to enable the feature representations of the attribute term extraction module and the sentiment classification module to learn from each other;

[0055] Step 2.1: Obtain the feature vector matrices of the input text and three sentiment resources, enhance the corresponding features in the input text in two stages using the sentiment resources, and predict the results of the enhanced text feature vectors;

[0056] Step 2.2: Combine the attribute terms and the sentiment tendency of each word obtained from the DP-GCN model and the Two-stage MSE model, and propose an interactive sharing unit ISU. The interactive sharing unit ISU is used to jointly learn two independent subtasks;

[0057] Step 3: Combine the obtained attribute term extraction task labels and sentiment classification task labels to obtain the final output result, that is: the attribute term - sentiment label pair, and obtain the final statement prediction result.

[0058] The attribute term extraction module is an attribute term extraction model based on dependency analysis and GCN. The attribute term extraction model uses a combination of a general domain vector and a specific domain vector as the vector representation of each word, and thus proposes an adjacency matrix construction method based on dependency analysis and PMI. The semantic features obtained through the LSTM network and the sentence structure features obtained through the GCN network are combined by weights to obtain the final vector representation of the input text.

[0059] The three sentiments in Step 2.1 are positive (POS), negative (NEG), neutral (NEU), and irrelevant word (0). The word vectors are represented by the BERT model, the sentence is represented by S, and the vector matrix of the sentence based on BERT is The vector matrix of sentiment words The vector matrix of intensifiers The vector matrix of negation words where t, m, k, p represent the length of the input text and the numbers of sentiment words, intensifiers, and negation words respectively, and d represents the dimension of the word vector.

[0060] The first-stage enhancement of the sentiment resources in the two-stage in Step 2.1 is used to establish the association matrix at the word level between the input text and the three sentiment resources. The enhancement in the first stage includes the following processing steps: Step 1: Obtain each association matrix through dot product operation; the calculation method formula is: M s =(w b ) T .w s ∈R t*m 、M i =(w b ) T .w i ∈R t*k 、M n =(w b ) T .w n ∈R t*p ; where: M sRepresents the association matrix between the input text and sentiment words. The elements in each row of the matrix represent the semantic similarity between each word in the input text and each sentiment word; M i and M n respectively represent the association matrix between the input text and intensity words and the association matrix between the input text and negation words. The meaning of each element in the matrix is similar to that of the elements in M s ;

[0061] Step 2: After obtaining the sentiment word association matrix M s , calculate its row average vector and column average vector respectively through average pooling and column average vector The specific calculation method is as follows: where and are each row vector and each column vector in the sentiment word association matrix M s respectively. Similarly, find the row average vector and column average vector of the intensity word association matrix M i and the negation word association matrix M n ;

[0062] Step 3: Combine the vector matrix W b of the input text in Step 2 and the column average vector of the sentiment word association matrix to calculate the input text X s enhanced by sentiment words, and calculate the sentiment word matrix b combining the input text information through W and the row average vector of the association matrix, and perform similar operations on intensity words and negation words in turn; the calculation method is as follows:

[0063] Step 4: The vector representation of the input text after being enhanced by three types of sentiment resources is the sum of the input text vector matrices related to the above three types of sentiment resources. The calculation method is as follows: X c = X s + X i + X n .

[0064] In Step 2.1, the second-stage enhancement of sentiment resources in the two-stage process is used to enhance the corresponding features of the hidden layer, which specifically includes the following steps: Step 1: After obtaining the input text feature matrix X c combining the information of three types of sentiment resources, input it into the LSTM network to obtain the hidden layer variable H c combining the context semantic information. The calculation method is as follows: H c = LSTM(X c );​

[0065] Step 2: Obtain the hidden layer H c Then, using the sentiment word matrix combined with the input text information and the degree word matrix combined with the input text information obtained in the first stage and the negation word matrix combined with the input text information again perform corresponding feature enhancement on the hidden layer matrix H of the input text; the calculation method is as follows: c where represents each column vector in the sentiment word matrix , then q s is obtained by performing a pooling operation on the sentiment matrix; represents the hidden layer vector of each word in the input text the weight score in the direction of the sentiment word of the entire sentence, u s and W s are parameter matrices, tanh represents the activation function; α i is the weight of each word after normalization; is the final vector of the word after being enhanced by the sentiment word, and the sentence enhanced by the sentiment word is represented as O s ;

[0066] Step 3: According to O in Step 2 s obtain the sentence representation O enhanced by the degree word and the sentence representation O enhanced by the negation word according to the calculation formula i , and the final vector representation of each sentence is the sum of the representations of the three sentences, and the calculation method is as follows: O = O n + O s + O i + O n .

[0067] The prediction of the result in Step 2.1 is reflected by the output module. After obtaining the final vector O of each word i , the feature vector of each word is converted into a label vector z = [p 1 , p 2 ,... p k through the fully connected layer. Each element p i in z represents that the word is the label c 1 in the label set class = [c 2 , c k ,... c i ​The probability, k represents the number of labeled tags. The prediction result maps each value in the label vector to the probability of the corresponding label through the softmax function, and then obtains the index of the maximum value in the probability vector through the argmax function. In the experimental dataset, the weighted cross-entropy loss function with L2 regularization term is adopted for the sentiment classification task. The weighted cross-entropy loss function with regularization term is used to solve the problem of unbalanced label quantities. The formula for calculating the loss of the sentiment classification model is: where y c is the One-Hot vector of the true label of the sample, w c is the weight of each category, μ is the L2 regularization coefficient, and Θ is the set of parameters.

[0068] Step 2.1 also includes the sentiment classification algorithm of Two-stage MSE in the corresponding feature enhancement of the input text in two stages. The algorithm is as follows:

[0069] Row1: Begin

[0070] Row2: Read each sentence text S in the data i

[0071] Row3: Let the number of iterations be N and the batch size be B;

[0072] Row4: For each S i in S

[0073] Row5: Obtain the vector representation W of the input sentence through the Bert model b

[0074] Row6: Obtain the vector representations W s 、W i 、W n

[0075] Row7: End for

[0076] Row8: Use the formula: M s =(w b ) T .w s ∈R t*m 、M i =(w b ) T .w i ∈R t*k 、M n =(w b ) T .w n ∈R t*p to calculate the sentiment word association matrix M s, Degree Word Association Matrix M i , Negation Word Association Matrix M n

[0077] Row9: Use the formula to obtain the input text X with enhanced sentiment words s , Input text X with enhanced degree words i , Input text X with enhanced negation words n and sentiment words combined with input text information Degree words combined with input text information Negation words combined with input text information

[0078] Row10: Use the formula X c = X s + X i + X n to calculate the sentence representation X after being enhanced by three types of sentiment resources in the first stage c

[0079] Row11: Input X c into LSTM, and use the formula H c = LSTM(X c ) to obtain the hidden layer vector representation

[0080] Row12: Use the formula to enhance the input text again respectively with

[0081] Row13: Use the formula O = O s + O i + O n to calculate the sentence representation O after sentiment word enhancement in the second stage

[0082] Row14: Obtain the sentiment polarity of each word in the sentence

[0083] Row15: For each n in N

[0084] Row16: For each b in B

[0085] Row17: Use the formula to calculate the error loss

[0086] Row18: Use the Adam optimization algorithm of gradient descent to update the network parameters

[0087] Row19: End for ​

[0088] Row 20: End for

[0089] Row 21: End.

[0090] The interactive sharing unit is used to calculate the correlation of the word feature representations in the tasks of two modules, and adjust the features of each word itself in combination with the correlation; specifically, it includes the following steps: Step 1: Calculate the correlation vector of each word in the attribute term extraction module and the sentiment classification module The calculation method formula is:[[]] where a and s represent the attribute term extraction task and the sentiment classification task respectively; represents the feature vector of word i obtained through the attribute term extraction task, represents the feature vector of word j obtained through the sentiment classification task, tanh is the activation function, and G is the parameter matrix; Step 2: After obtaining the correlation vector of word i and word j, transform it into a weight score through the formula ; where is the parameter vector, is the association score; the set of correlation vectors between words is represented by the matrix S a-s and S s-a ; Step 3: Enhance the feature matrices of the calculation attribute term extraction module and the sentiment classification module respectively through the correlation matrix; the calculation method formula is: H a = H a + softmax(S a-s )H s , H s = H s + softmax(S s-a )H a ; where H a and H s are the sentence representations enhanced by the interactive sharing unit, and are used to predict the label pair of the statement in Step 3 after being processed by the output module.

[0091] The output module uses the DP-GCN model for output. The sentiment classification model has two selection methods for multi-word attribute terms: The first case: When the sentiment tendencies of the words forming the same attribute term are inconsistent, the majority principle is used; The second case: When the sentiment tendencies are inconsistent and the occurrences of various sentiments are the same, the sentiment that appears first is selected.

[0092] To verify the effectiveness of the model proposed in this solution and make it comparable to other attribute term and sentiment joint extraction tasks, this solution will use the international semantic evaluation competition SemEval dataset and the Twitter dataset commonly used in this task;

[0093] Among them, the SemEval dataset consists of restaurant reviews and laptop reviews. For the restaurant review dataset, the competition data from 2014 - 2016 is used (hereinafter referred to as Rest14 - 16), and for the laptop review dataset, the competition data from 2014 is used (hereinafter referred to as Lap14).

[0094] Each dataset is divided into a training set and a test set, and the specific information is shown in the following table:

[0095]

[0096]

[0097] This solution uses accuracy, precision, recall, and F1 - score as the evaluation criteria for the sentiment classification model based on Two - stage MSE and the interactive attribute term and sentiment joint extraction model:

[0098] Accuracy refers to the proportion of the number of samples correctly classified by the model to the total number of all samples, and is usually used to evaluate the overall accuracy of the model. The calculation formula is:

[0099] Precision, also known as the precision rate, refers to the proportion of samples actually belonging to a certain category among the samples predicted by the model as belonging to this category. The calculation formula is:

[0100] Recall, also known as the recall rate, refers to the proportion of the number of samples correctly predicted in a certain category to the total number of samples in this category. The calculation formula is:

[0101] TP represents that positive examples are correctly judged as positive examples; FP represents that negative examples are wrongly judged as positive examples, also known as false positive examples; FN represents that positive examples are wrongly judged as negative examples, also known as false negative examples; TN represents that negative examples are correctly judged as negative examples;

[0102] The F1 - score combines precision and recall and can comprehensively evaluate the performance of the model. Numerically, it is equal to the harmonic mean of precision and recall. The calculation formula is:

[0103] To verify the performance of the Two-stage MSE sentiment classification model proposed in this solution, CNN and LSTM are used as baseline models for comparison in this solution. The evaluation metrics are accuracy and F1 value. "w / os", "w / oi", and "w / on" in the table respectively indicate removing sentiment words, intensity words, and negation words from the sentiment resources used. The experimental results are shown in the following table:

[0104]

[0105] It can be seen from the data in the table that the model proposed in this solution has a substantial performance improvement compared to traditional neural network models such as LSTM and CNN. The F1 value on the Lap14 dataset has increased by 3.47% compared to DE-LSTM, indicating that integrating sentiment resources into the neural network is an effective method to improve sentiment classification performance;

[0106] The experimental data in the last three rows of the table are ablation experiments on the Two-stage MSE model proposed in this chapter. The F1 values of the ablation experiments in the above data table are plotted as a histogram, and the results are as Figure 2 shown:

[0107] Referring to Figure 2 it can be known that removing sentiment words has the greatest impact on the model, and the F1 value has decreased by 3.18% on the Lapt14 dataset; removing negation words has the least impact on the model, and it has decreased by 1.4% on the Lap14 dataset. The reason may be that the number of negation words used in this paper is too small, and its role in model training is limited.

[0108] To evaluate the role of the interactive sharing unit proposed in this solution in the model, it is removed for experiments and compared with the effect of the complete model. The experiments are carried out on three datasets, and the evaluation metric is the F1 value. The results are shown in the following table:

[0109]

[0110] It can be known from the above table that the performance of the model has decreased significantly after removing the interactive sharing unit. The performance of the Twitter dataset has decreased by up to 5.61%, indicating that the interactive sharing unit can effectively enable the two subtasks to provide information to each other and improve the overall performance of the model.

[0111] The above specific embodiments are only several preferred embodiments of the present invention. Based on the technical solution of the present invention and the relevant inspirations of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.

Claims

1. Method for an interactive attribute term and sentiment joint extraction model, Characterized in that, It includes an attribute term extraction module, a sentiment classification module, and an interaction sharing unit, and specifically includes the following generation steps: Step 1: A neural network combined with external sentiment resources obtains the sentiment tendency of each word in the sentence; Step 2: The attribute term extraction module is combined with the sentiment classification module, and an interaction sharing unit is used to enable the feature representations of the attribute term extraction module and the sentiment classification module to learn from each other; Step 2.1: Obtain the feature vector matrix of the input text and three sentiment resources, use the sentiment resources to enhance the corresponding features in the input text in two stages, and predict the results of the enhanced text feature vectors; Step 2.2: Combine the attribute terms in the sentence and the sentiment tendency of each word obtained by the DP-GCN model and the Two-stage MSE model, and propose an interaction sharing unit ISU. The interaction sharing unit ISU is used to jointly learn two independent subtasks. The sentiment classification algorithm of the Two-stage MSE model is: Row1: Begin Row2: Read each sentence of text S in the data i Row3: Let the number of iterations be N, and the batch size be B; Row4: For each S i in S Row5: Obtain the vector representation W of the input sentence through the Bert model b Row6: Obtain the vector representations W of sentiment words, degree words, and negation words through the Bert model s , W i , W n Row7: End for Row8: Use the formula: M s =(w b ) T .w s ∈R t*m 、M i =(w b ) T .w i ∈R t*k 、M n =(w b ) T .w n ∈R t*p Calculate the sentiment word association matrix M s 、the intensity word association matrix M i 、the negation word association matrix M n Row9: Use the formula to obtain the input text X with enhanced sentiment words s , the input text X with enhanced intensity words i , the input text X with enhanced negation words n and the sentiment words combined with the input text information the intensity words combined with the input text information the negation words combined with the input text information Row10: Use formula X c = X s + X i + X n Calculate the sentence representation X enhanced by three emotional resources in the first stage c Row11: Input X c into the LSTM, and use the formula H c = LSTM(X c ) to obtain the hidden layer vector representation Row12: Use the formula respectively use Enhance the input text again Row13: Use the formula O = O s +O i +O n Calculate the sentence representation O after enhancing the sentiment words in the second stage Row14: Obtain the sentiment polarity of each word in the sentence Row15: For each n in N Row16: For each b in B Row 17: Use the formula to calculate the error loss Row18: Use the gradient descent optimization algorithm Adam to update the network parameters Row19: End for Row20: End for Row21: End; Step 3: Combine the obtained attribute term extraction task labels and sentiment classification task labels to obtain the final output result, that is, the attribute term - sentiment label pair, and obtain the final sentence prediction result.

2. The method for an interactive attribute term and sentiment joint extraction model according to claim 1, Characterized in that, The attribute term extraction module is an attribute term extraction model based on dependency analysis and GCN. The attribute term extraction model uses a combination of a general domain vector and a specific domain vector as the vector representation of each word, thereby proposing a method for constructing an adjacency matrix based on dependency analysis and PMI. The semantic features obtained through the LSTM network and the sentence structure features obtained through the GCN network are combined by weights to obtain the final vector representation of the input text.

3. The method for an interactive attribute term and sentiment joint extraction model according to claim 1, Characterized in that, The three emotions in step 2.1 are positive, negative, neutral, and irrelevant words. The word vectors are represented by the BERT model, the sentence is represented by S, and the vector matrix of the sentence based on BERT is Vector matrix of emotion words Vector matrix of intensity words Vector matrix of negation words Among them, t, m, k, and p represent the length of the input text and the numbers of emotion words, intensity words, and negation words respectively, and d represents the dimension of the word vector.

4. The method for an interactive attribute term and sentiment joint extraction model according to claim 3, Characterized in that, In the first stage of the two-stage sentiment resource enhancement in step 2.1, it is used to establish an association matrix between the input text and the three sentiment resources at the word level. The enhancement in the first stage includes the following processing steps: Step 1: Obtain each association matrix through dot product operation; the calculation method formula is: M s =(w b ) T .w s ∈R t*m 、M i =(w b ) T .w i ∈R t*k 、M n =(w b ) T .w n ∈R t*p ; where: M s represents the association matrix between the input text and the sentiment words, and the elements in each row of the matrix represent the semantic similarity between each word in the input text and each sentiment word; M i and M n represent the association matrix between the input text and the degree words and the association matrix between the input text and the negation words respectively, and the meaning of each element in the matrix is similar to that of the elements in M s . Step 2: Emotional Word Association Matrix M s After obtaining it, calculate its row average vector through average pooling respectively and column average vector The specific calculation method is as follows: where and are each row vector and each column vector in the emotional word association matrix M s respectively. Similarly, find the row average vector and column average vector of the intensity word association matrix M i and the negation word association matrix M n ; Step 3: Combine the vector matrix W of the input text obtained in Step 2 b and the column average vector of the sentiment word association matrix to calculate the input text X enhanced by sentiment words s , and calculate the sentiment word matrix incorporating the input text information through the row average vector of the association matrix b and W , and perform similar operations on degree words and negation words in sequence; the calculation method is as shown in the formula: ​ Step 4: The vector representation of the input text after being enhanced by the three emotion resources is the sum of the input text vector matrices related to the above three emotion resources, and the calculation method is as shown in the formula: X c = X s + X i + X n .

5. The method for an interactive attribute term and sentiment joint extraction model according to claim 4, Characterized in that, In step 2.1, the enhancement of the emotional resources in the second stage of the two stages is used to enhance the corresponding features of the hidden layer, which specifically includes the following steps: Step 1: After obtaining the input text feature matrix X that combines the information of the three emotional resources c it is input into the LSTM network to obtain the hidden layer variable H that combines the context semantic information c , and the calculation method is as follows: H c = LSTM(X c ); Step 2: Obtain the hidden layer H c Then, use the sentiment word matrix combined with the input text information and the degree word matrix combined with the input text information obtained in the first stage and the negation word matrix combined with the input text information to perform corresponding feature enhancement on the hidden layer matrix H of the input text again c ; The calculation method is as follows: where represents each column vector in the sentiment word matrix , then q s is obtained by performing a pooling operation on the sentiment matrix; represents the hidden layer vector of each word in the input text is the weight score of the sentiment word direction of each word in the whole sentence, u s and W s are parameter matrices, and tanh represents the activation function; α i is the weight of each word after normalization; is the final vector of the word after sentiment word enhancement, and the sentence after sentiment word enhancement is represented as O s ; Step 3: According to the O in Step 2 s Obtain the sentence representation O enhanced by degree words according to the calculation formula i And the sentence representation O enhanced by negative words n , The final vector representation of each sentence is the sum of the representations of the three sentences, and the calculation method is as the formula: O = O s + O i + O n .

6. The method for an interactive attribute term and sentiment joint extraction model according to claim 5, Characterized in that, The prediction of the result in step 2.1 is reflected by the output module. After obtaining the final vector of each word O i the feature vector of each word is converted into a label vector z = [p 1 , p 2 ,... p k through a fully connected layer. Each element p i in z represents the probability that the word is a label c 1 in the label set class = [c 2 , c k ,... c i . k represents the number of labeled tags. The prediction result maps each value in the label vector to the probability of the corresponding label through the softmax function, and then obtains the index of the maximum value in the probability vector through the argmax function. In the experimental dataset, the weighted cross-entropy loss function with L2 regularization term is adopted for the sentiment classification task. The weighted cross-entropy loss function with regularization term is used to solve the problem of unbalanced label quantity. The formula for the loss calculation method of the sentiment classification model is: where y c is the One-Hot vector of the true label of the sample, w c is the weight of each category, μ is the L2 regularization coefficient, and Θ is the set of parameters.

7. The method of the interactive attribute term and sentiment joint extraction model according to claim 1, characterized in that, The interactive sharing unit is used to calculate the correlation between the word feature representations in the tasks of two modules, and adjust the features of each word itself in combination with the correlation; specifically, it includes the following steps: Step 1: Calculate the correlation vector of each word in the attribute term extraction module and the sentiment classification module The calculation method formula is: where a and s represent the attribute term extraction task and the sentiment classification task respectively; represents the feature vector of word i obtained through the attribute term extraction task, represents the feature vector of word j obtained through the sentiment classification task, tanh is the activation function, and G is the parameter matrix; Step 2: After obtaining the correlation vector of word i and word j, transform it into a weight score through the formula ; where is the parameter vector, is the association score; the set of correlation vectors between words is represented by the matrix S a-s and S s-a ; Step 3: Enhance the feature matrices of the calculation attribute term extraction module and the sentiment classification module respectively through the correlation matrix; the calculation method formula is: H a =H a +softmax(S a-s )H s , H s =H s +softmax(S s-a )H a ; where H a and H s are the sentence representations enhanced by the interactive sharing unit, and are used to predict the label pair of the statement in Step 3 after being processed by the output module.

8. The method of the interactive attribute term and sentiment joint extraction model according to claim 7, characterized in that, the output module outputs using the DP-GCN model, and the sentiment classification model has two selection methods for multi-word attribute terms: the first case: when the sentiment tendencies of the words forming the same attribute term are inconsistent, the majority principle is used; the second case: when the sentiment tendencies are inconsistent and the occurrences of various sentiments are the same, the sentiment that appears first is selected.

Images