A sentiment prediction method and system based on interactive dual graph convolutional network

Abstract

The present invention provides a sentiment prediction method and system based on an interactive dual graph convolutional network, comprising: obtaining a text to be predicted and converting it into a plurality of word vector sequences; extracting context information of the plurality of word vector sequences to obtain a hidden state vector; based on the plurality of word vector sequences, using a syntactic graph convolutional network to obtain a syntactic graph; based on the hidden state vector, using a semantic graph convolutional network to obtain a semantic graph; based on the syntactic graph and the semantic graph, interactively learning syntactic information and semantic information to obtain a syntactic graph and a semantic graph after interactive learning; based on the syntactic graph and the semantic graph after interactive learning, predicting the sentiment probability distribution of the text to be predicted. The accuracy and stability of the judgment of sentiment polarity are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a sentiment prediction method and system based on an interactive dual graph convolutional network. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of social media, the number of online comments has also exploded. More and more people are willing to express their attitudes and emotions on the Internet, rather than simply browsing and accepting. A large amount of online comment data is often accompanied by the commentator's emotional information, such as "happy", "angry", "frustrated", etc. Since the user comment data appearing on the Internet contains the user's emotional information, individual consumers and companies have a positive attitude towards whether online comments have an important impact on purchase decisions, so people hope to obtain valuable information by analyzing and processing the comment data contained in the Internet. Therefore, sentiment analysis has important academic value and broad practical significance in life.

[0004] Traditional sentiment analysis methods mainly analyze sentiment tendencies for the entire sentence or the entire document. However, users usually express different sentiments on different entities or aspects in the same sentence. For example, in "This restaurant is so delicious, but the waiter's attitude is too cold", traditional sentiment analysis cannot accurately judge the sentiment tendency. Therefore, aspect-level sentiment analysis came into being. Aspect-level sentiment analysis predicts the sentiment classification of a specific target in the text. Sentiment analysis is divided into coarse-grained sentiment analysis methods based on sentence level and document level and fine-grained sentiment analysis methods based on aspect level according to different granularity. The purpose of aspect-level sentiment analysis task is to infer the sentiment polarity of each aspect in this text. Sentiment polarity is generally divided into three types: positive, negative and neutral. In the above example, the sentiment polarity for "taste" is positive, and the sentiment polarity for "service" is negative. Because aspect-level sentiment analysis can provide more comprehensive and accurate information in applications, it has become the most popular and active subtask in sentiment analysis in recent years.

[0005] With the introduction of aspect-level sentiment analysis into graph neural network models, good results have been achieved, which has largely made up for the shortcomings of traditional methods. However, there are still some defects in these studies using graph neural networks. There are a large number of syntactic structure errors, informal expressions and complexity of online comments in the comments. Summary of the invention

[0006] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a sentiment prediction method and system based on an interactive dual graph convolutional network, which improves the accuracy and stability of sentiment polarity judgment.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A first aspect of the present invention provides a sentiment prediction method based on an interactive dual graph convolutional network, comprising:

[0009] Get the text to be predicted and convert it into several word vector sequences;

[0010] Extract the context information of several word vector sequences to obtain the hidden state vector;

[0011] Based on several word vector sequences, a syntactic graph convolutional network is used to obtain a syntactic graph;

[0012] Based on the hidden state vector, a semantic graph convolutional network is used to obtain a semantic graph;

[0013] Based on the syntactic graph and the semantic graph, interactively learn the syntactic information and the semantic information to obtain the syntactic graph and the semantic graph after interactive learning;

[0014] Based on the syntactic graph and semantic graph after interactive learning, the sentiment probability distribution of the text to be predicted is predicted.

[0015] Furthermore, the specific steps of obtaining the syntactic graph are:

[0016] Based on several word vector sequences, calculate the aspect-oriented syntactic dependency adjacency matrix;

[0017] The grammatical encoding of the adjacency matrix is ​​adopted, and the hidden state vector is used as the initial node representation to obtain a syntactic graph.

[0018] Furthermore, the specific steps of calculating the aspect-oriented syntactic dependency adjacency matrix based on a plurality of word vector sequences are as follows:

[0019] Generate dependency graphs and weight dependency graphs based on several word vector sequences;

[0020] The dependency graph and weighted dependency graph are integrated to obtain the aspect-oriented syntactic dependency adjacency matrix.

[0021] Furthermore, the specific steps of predicting the sentiment probability distribution of the text to be predicted are:

[0022] After average pooling, the syntactic graph and the semantic graph after the interactive learning are respectively connected to obtain the final features;

[0023] After the final features are fed into a linear layer, an activation function is used to generate the sentiment probability distribution.

[0024] Furthermore, the semantic graph convolutional network generates and updates the semantic graph through a multi-head self-attention mechanism.

[0025] Furthermore, the context information of the plurality of word vector sequences is extracted using a bidirectional LSTM network.

[0026] Furthermore, the GloVe word embedding tool is used to convert the text to be predicted into a sequence of word vectors.

[0027] A second aspect of the present invention provides a sentiment prediction system based on an interactive dual graph convolutional network, comprising:

[0028] A text conversion module is configured to: obtain the text to be predicted and convert it into a number of word vector sequences;

[0029] A context information extraction module is configured to: extract context information of a plurality of word vector sequences to obtain a hidden state vector;

[0030] A syntactic graph convolution module is configured to: obtain a syntactic graph based on a plurality of word vector sequences using a syntactic graph convolution network;

[0031] A semantic graph convolution module is configured to: obtain a semantic graph using a semantic graph convolution network based on the hidden state vector;

[0032] An interactive learning module is configured to: interactively learn syntactic information and semantic information based on the syntactic graph and the semantic graph, and obtain the syntactic graph and the semantic graph after interactive learning;

[0033] The sentiment probability distribution prediction module is configured to predict the sentiment probability distribution of the text to be predicted based on the syntactic graph and semantic graph after interactive learning.

[0034] The third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-mentioned method for emotion prediction based on an interactive dual graph convolutional network.

[0035] The fourth aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the emotion prediction method based on an interactive dual graph convolutional network as described above are implemented.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention provides a sentiment prediction method based on an interactive dual graph convolutional network, which realizes coordinated optimization between the syntactic graph and the semantic graph by interactive learning between the two, solves the problems of a large number of syntactic structure errors, informal expressions and complexity in online reviews, and improves the accuracy and stability of sentiment polarity judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0039] Figure 1 is a flow chart of a sentiment analysis prediction method according to Embodiment 1 of the present invention;

[0040] Figure 2 This is an interactive dual graph convolutional network model diagram of embodiment 1 of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0042] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0043] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0044] Embodiment 1

[0045] This embodiment provides a sentiment prediction method based on an interactive dual graph convolutional network. Figure 1 As shown, the specific steps include:

[0046] Step 1: Get the text to be predicted, perform word embedding training on the text to be predicted, and convert the text into word vector embedding, that is, obtain the word vector sequence of each word in the text.

[0047] Specifically, we use the GloVe word embedding tool. Given a context sequence consisting of m words and aspect sequences consisting of n words Among them, W a It is Wc Subsequence of . Through the pre-trained GloVe embedding matrix Among them, d m represents the embedding dimension of the word vector, and |V| represents the size of the vocabulary.

[0048] Step 2: Convert the word vector sequences obtained in step 1 into context representation, i.e., hidden state vector H c ={h 1 ,h 2 ,...,h n Specifically, several word vector sequences are input into a bidirectional LSTM (BiLSTM) network to capture the long-distance dependencies in the sentence and extract the text context information. That is, for the above embedding, a bidirectional LSTM is used to obtain the hidden state vector to obtain the context representation of the word embedding.

[0049] Specifically, we use BiLSTM as a sentence encoder to extract hidden context representations. We embed the words of the sentence into BiLSTM to generate a hidden state vector H c ={h 1 ,h 2 ,...,h n},in is the hidden state vector of BiLSTM at time t, and the hidden state vector d lstm The dimension of the output is unidirectional LSTM, 2d lstm Indicates the dimension of the bidirectional LSTM.

[0050] Step 3: Based on several word vector sequences, calculate the aspect-oriented syntactic dependency adjacency matrix. Input the word embeddings from step 1 into the Spacy parser to generate a dependency tree, which is then used to generate a general graph, and the resulting graph is refined to obtain a graph for a specific aspect.

[0051] Each input sentence generates a common dependency graph on the dependency tree, and the adjacency matrix D derived from the dependency tree of the input sentence is i,j :

[0052]

[0053] In order to highlight specific aspects in the context words, the graph is optimized by calculating the relative position weight of each element of the adjacency matrix, and an aspect-centric enhanced dependency graph (weighted dependency graph) is obtained.

[0054]

[0055] Among them, p s is the position where the aspect word starts, w i and wj are context words, It is a collection of aspect words.

[0056] In order to enhance the grammatical dependency of context words and generate the relationship between aspect words and context words, the aspect-oriented weighted dependency graph and the ordinary dependency graph are integrated to obtain the aspect-oriented syntactic dependency adjacency matrix G i,j :

[0057]

[0058] Step 4: Figure 2 As shown in Figure 2, the outputs of step 2 and step 3 are applied to the graph convolution network to obtain the Syntax Graph Convolution Networks (SynGCN). i,j The grammatical encoding of the hidden state vector H in BiLSTM c As the initial node representation in the syntactic graph, we can get the syntactic graph representation H syn , and use Shows the hidden representation of all facet nodes:

[0059]

[0060] Among them, w syn and are the weight matrix and bias respectively, and σ represents the activation function.

[0061] Step 5: Figure 2 As shown in the figure, the context output of step 2, that is, the hidden state vector, is input into the semantic graph convolution network (SemGCN), which is generated and updated through a multi-head self-attention mechanism, focusing on extracting the most relevant information in the semantic space to obtain a semantic graph.

[0062] Initialization: Use the multi-head self-attention mechanism to get K attention score matrices. In order to improve robustness, sum the K matrices and then select the top-k words, so that the top-k important context words are retained. Initialization is as follows:

[0063]

[0064]

[0065]

[0066]

[0067] in, is the i-th adjacency matrix, represents the top k large adjacency matrices selected before entering the first layer of GCN, and is a learnable matrix, is the deviation term, d lstm Represents the hidden state dimension of lstm, and the top-k() function means finding the top k largest data from the data.

[0068] Update: Unlike initialization, the input It is composed of the outputs generated in layers 0 to l-1 in series. The multi-head self-attention mechanism generates k attention score matrices. As with the initialization, the softmax function is used to calculate the matrix with the highest probability, and the top-k selection is performed. The GCN layer is used to extract deep semantic information. The output of the last layer of SemGCN is represented as H sem , i.e., the semantic graph, contains the most relevant semantic information of aspects and their opinion words, and uses Represents the hidden representation of all aspect nodes. Updated as follows:

[0069]

[0070]

[0071]

[0072]

[0073]

[0074] in, and is a learnable matrix, is the deviation term, is the output of the bidirectional LSTM.

[0075] Step 6: The outputs of step 4 and step 5, namely the syntactic graph and the semantic graph, are embedded in the interactive attention block for interactive learning. They are cross-referenced in two specific graph convolutional networks to interactively learn the syntactic information and the semantic information, and the outputs after interactive learning, namely the syntactic graph and the semantic graph H after interactive learning, are obtained respectively. syn ' and H sem '.

[0076] Step 7: Based on the syntactic graph and semantic graph after interactive learning, predict the sentiment probability distribution of the text to be predicted.

[0077] Specifically, the output of step 6, i.e., the syntactic graph and semantic graph after interactive learning, are respectively subjected to the average pooling function and then concatenated. The obtained representation information is then sent to a linear layer, and then the activation function (softmax function) is used to generate the sentiment probability distribution to obtain the final sentiment polarity prediction result.

[0078] Applying average pooling and concatenation operations on the aspect nodes of SynGCN and SemGCN, we obtain the final feature representation for the aspect-level sentiment analysis task:

[0079]

[0080]

[0081]

[0082] where f(·) is the average pooling function applied to the aspect node representation.

[0083] The final feature r is fed into a linear layer, and then the softmax function is used to generate the emotional probability distribution to obtain the final emotional polarity, that is:

[0084] p(a)=softmax(W P r+b p ) (17)

[0085] Among them, W P and b p are the learnable weights and biases respectively.

[0086] The present invention solves the problems of a large number of syntactic errors, informal expressions and complexity of online comments in comments. Syntax is a supplement to semantics. Based on this basic assumption, an interactive dual graph convolutional network is designed to overcome the above shortcomings. First, in order to use the information in the semantic space in a balanced way, a multi-head self-attention mechanism is applied in the graph convolutional network to generate and update the semantic graph. Secondly, the dependency tree of the sentence is converted into a graph, and the graph is refined to generate a syntactic graph. Using the semantic graph and the syntactic graph, two specific graph convolutional networks are used to extract specific information from the semantic and syntactic spaces. Finally, the interactive attention block is used to interactively learn the information of the syntactic graph convolutional network and the semantic graph convolutional network. By using the interactive attention block between the syntactic graph and the semantic graph, the coordinated optimization between the two is achieved. The accuracy and stability of the judgment of sentiment polarity on different data sets are improved.

[0087] Embodiment 2

[0088] This embodiment provides a sentiment prediction system based on an interactive dual graph convolutional network, which specifically includes the following modules:

[0089] A text conversion module is configured to: obtain the text to be predicted and convert it into a number of word vector sequences;

[0090] A context information extraction module is configured to: extract context information of a plurality of word vector sequences to obtain a hidden state vector;

[0091] A syntactic graph convolution module is configured to: obtain a syntactic graph based on a plurality of word vector sequences using a syntactic graph convolution network;

[0092] A semantic graph convolution module is configured to: obtain a semantic graph using a semantic graph convolution network based on the hidden state vector;

[0093] An interactive learning module is configured to: interactively learn syntactic information and semantic information based on the syntactic graph and the semantic graph, and obtain the syntactic graph and the semantic graph after interactive learning;

[0094] The sentiment probability distribution prediction module is configured to predict the sentiment probability distribution of the text to be predicted based on the syntactic graph and semantic graph after interactive learning.

[0095] It should be noted here that each module in this embodiment corresponds to each step in Example 1 one by one, and the specific implementation process is the same, which will not be repeated here.

[0096] Embodiment 3

[0097] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the emotion prediction method based on an interactive dual graph convolutional network as described in the first embodiment above are implemented.

[0098] Embodiment 4

[0099] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the emotion prediction method based on an interactive dual graph convolutional network as described in the first embodiment are implemented.

[0100] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0101] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0102] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0104] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0105] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A sentiment prediction method based on interactive dual graph convolutional network, characterized in that: include: Get the text to be predicted and convert it into several word vector sequences; Extract the context information of several word vector sequences to obtain the hidden state vector; Based on several word vector sequences, a syntactic graph convolutional network is used to obtain a syntactic graph; Based on the hidden state vector, a semantic graph convolutional network is used to obtain a semantic graph; Based on the syntactic graph and the semantic graph, interactively learn the syntactic information and the semantic information to obtain the syntactic graph and the semantic graph after interactive learning; Based on the syntactic graph and semantic graph after interactive learning, the sentiment probability distribution of the text to be predicted is predicted; The specific steps of obtaining the syntactic graph are: Based on several word vector sequences, calculate the aspect-oriented syntactic dependency adjacency matrix; Using the syntax encoding of the adjacency matrix, taking the hidden state vector as the initial node representation, to obtain a syntax graph; The specific steps of calculating the aspect-oriented syntactic dependency adjacency matrix based on a plurality of word vector sequences are as follows: Generate dependency graphs and weight dependency graphs based on several word vector sequences; Integrate the dependency graph and the weighted dependency graph to obtain the aspect-oriented syntactic dependency adjacency matrix; The semantic graph convolutional network generates and updates the semantic graph through a multi-head self-attention mechanism; The multi-head self-attention mechanism is used to obtain K attention score matrices, the K matrices are summed, and then the top-k important context words are retained and initialized as follows: （5） （6） （7） （8） in, is the i-th adjacency matrix, represents the top k large adjacency matrices selected before entering the first layer of GCN, and is a learnable matrix, is the deviation term, d lstm express LSTM The hidden state dimension of top-k The function means finding the first k largest data from the data. is the hidden state vector; Based on several word vector sequences, we calculate the aspect-oriented syntactic dependency adjacency matrix, input the word embeddings into the Spacy parser to generate a dependency tree, and then use the dependency tree to generate a general graph, and then refine the obtained graph to obtain a specific aspect graph; Each input sentence generates a normal dependency graph on the dependency tree, and the adjacency matrix derived from the dependency tree of the input sentence is : （1）； In order to highlight specific aspects in the context words, the graph is optimized by calculating the relative position weight of each element of the adjacency matrix to obtain an aspect-centric enhanced dependency graph. : （2） in, is the starting position of the aspect word, wi and wj are context words, is a collection of aspect words; In order to enhance the grammatical dependency of context words and generate the relationship between aspect words and context words, the aspect-oriented weighted dependency graph and the ordinary dependency graph are integrated to obtain the aspect-oriented syntactic dependency adjacency matrix. : （3）； Unlike initialization, the input From layer 0 to The outputs generated in are concatenated, and the multi-head self-attention mechanism generates k attention score matrices. As with the initialization, the softmax function is used to calculate the matrix with the highest probability, and top-k selection is performed. The GCN layer is used to extract deep semantic information. The output of the last layer of SemGCN is represented as , i.e., the semantic graph, contains the most relevant semantic information of aspects and their opinion words, and uses The hidden representation of all aspect nodes is updated as follows: （9） （10） （11） （12） （13） in, and is a learnable matrix, is the deviation term, is the output of the bidirectional LSTM.

2. The emotion prediction method based on interactive dual graph convolutional network as claimed in claim 1, characterized in that: The specific steps of predicting the sentiment probability distribution of the text to be predicted are: After average pooling, the syntactic graph and the semantic graph after the interactive learning are respectively connected to obtain the final features; After the final features are fed into a linear layer, an activation function is used to generate the sentiment probability distribution.

3. The emotion prediction method based on interactive dual graph convolutional network according to claim 1, characterized in that: The context information of the word vector sequences is extracted by using a bidirectional LSTM network.

4. The emotion prediction method based on interactive dual graph convolutional network according to claim 1, characterized in that: The GloVe word embedding tool is used to convert the text to be predicted into a sequence of word vectors.

5. A sentiment prediction system based on an interactive dual graph convolutional network, using the sentiment prediction method based on an interactive dual graph convolutional network as described in any one of claims 1 to 4, characterized in that: include: A text conversion module is configured to: obtain the text to be predicted and convert it into a number of word vector sequences; A context information extraction module is configured to: extract context information of a plurality of word vector sequences to obtain a hidden state vector; A syntactic graph convolution module is configured to: obtain a syntactic graph based on a plurality of word vector sequences using a syntactic graph convolution network; A semantic graph convolution module is configured to: obtain a semantic graph using a semantic graph convolution network based on the hidden state vector; An interactive learning module is configured to: interactively learn syntactic information and semantic information based on the syntactic graph and the semantic graph, and obtain the syntactic graph and the semantic graph after interactive learning; The sentiment probability distribution prediction module is configured to predict the sentiment probability distribution of the text to be predicted based on the syntactic graph and semantic graph after interactive learning.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the emotion prediction method based on an interactive dual graph convolutional network as described in any one of claims 1 to 4 are implemented.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the emotion prediction method based on an interactive dual graph convolutional network as described in any one of claims 1-4 are implemented.

Images