A method and device for sentiment classification of movie reviews based on graph neural network

Abstract

The present invention discloses a method and device for sentiment classification of film reviews based on graph neural network. Based on the grammatical information of text sentences and the window co-occurrence relationship of text words, the text sentences are constructed into a graph data structure, the adjacency matrix of the text co-occurrence graph and the syntactic dependency graph are obtained respectively, the word embedding vector matrix and the adjacency matrix are further input into the graph convolutional network module to obtain a weighted feature matrix, and a hybrid pooling module is used to perform pooling operation on the weighted feature matrix of the text co-occurrence graph and the weighted feature matrix of the syntactic dependency graph to obtain the feature vector of the text co-occurrence graph and the feature vector of the syntactic dependency graph, and the feature vectors are fused and input into the classifier to obtain the classification result. The present invention enables the learned network representation to contain not only the information of neighboring words, but also the information of relatively distant words related by grammatical relationship, thereby improving the accuracy of sentiment classification.

Description

Technical Field

[0001] The present application belongs to the field of natural language processing technology, and in particular, relates to a method and device for sentiment classification of movie reviews based on graph neural networks. Background Art

[0002] We are now in an era of the Internet and big data. Every day, a large amount of data is generated in various forms, such as text, voice, video, etc. Text data accounts for a large part of it. With the rapid development of the cultural industry, the number of movies released each year is constantly increasing, and with it comes a variety of movie reviews. For example, well-known movie websites such as Douban and Maoyan contain a large number of movie reviews. How to mine useful information from these movie reviews has become a key issue, and an important step is to classify the emotions of these movie reviews.

[0003] Sentiment classification is a basic and important task in natural language processing. In the early days, sentiment classification mainly used traditional machine learning methods, which first performed feature engineering and finally classified feature vectors through classifier methods, such as support vector machine models, k-nearest neighbor methods, naive Bayes, etc. However, such methods rely too much on manual feature extraction and are relatively inefficient. With the progress of deep learning research, the word2vec and Glove word vector models were proposed, and deep learning began to be applied to the field of natural language processing, which eliminated the tedious steps of manually extracting text features, and applied convolutional neural networks and recurrent neural networks to sentiment classification tasks. Then researchers proposed models such as TextCNN and CharCNN, and achieved good results. However, both convolutional neural networks and recurrent neural networks only focus on the sequence model of text, ignoring the structure between sentiment texts, while the graph convolutional neural network GCN can retain structural information in the process of learning text embedding, thereby further improving the sentiment classification effect of movie reviews. Summary of the invention

[0004] The purpose of this application is to provide a method and device for sentiment classification of movie reviews based on graph neural networks, so as to overcome the problem that existing sentiment classification methods ignore the grammatical characteristics of text and the topological structure of text, enrich the embedded information of text, further optimize the embedded representation of sentiment in text, and improve the accuracy of movie review classification.

[0005] In order to achieve the above purpose, the technical solution of this application is as follows:

[0006] A method for sentiment classification of movie reviews based on a graph neural network, the method for sentiment classification of movie reviews based on a graph neural network comprising:

[0007] Collect movie review text datasets, preprocess each movie review text to obtain a text vocabulary, represent the words in the text vocabulary with embedding vectors, and obtain the word embedding vector matrix of the movie review text;

[0008] The words in the text vocabulary are used as nodes, and the co-occurrence relationship between words is used as the edge to construct a text co-occurrence graph, and the adjacency matrix of the text co-occurrence graph is obtained;

[0009] The words in the text vocabulary are used as nodes and the syntactic relations are used as edges to construct the syntactic dependency graph of the text, and the adjacency matrix of the syntactic dependency graph is obtained;

[0010] Input the word embedding vector matrix and the adjacency matrix of the text co-occurrence graph into the text co-occurrence graph convolutional network module to obtain the weighted feature matrix of the text co-occurrence graph;

[0011] Input the word embedding vector matrix and the adjacency matrix of the syntactic dependency graph into the syntactic dependency graph convolutional network module to obtain the weighted feature matrix of the syntactic dependency graph;

[0012] A hybrid pooling module is used to perform pooling operations on the weighted feature matrix of the text co-occurrence graph and the weighted feature matrix of the syntactic dependency graph to obtain the feature vector of the text co-occurrence graph and the feature vector of the syntactic dependency graph;

[0013] Adopting the adaptive feature fusion module to adaptively fuse the feature vectors of the text co-occurrence graph and the feature vectors of the syntactic dependency graph;

[0014] The adaptively fused features are input into the text classifier module to predict the sentiment classification results of the movie review text.

[0015] Furthermore, the word embedding vector matrix and the adjacency matrix of the text co-occurrence graph are input into the text co-occurrence graph convolutional network module to obtain the weighted feature matrix of the text co-occurrence graph, including:

[0016] First, input the word embedding vector matrix and the adjacency matrix of the text co-occurrence graph into the text co-occurrence graph convolutional network module to obtain the feature matrix of the text co-occurrence graph:

[0017] H=[h1,…,h i ,…,h s ] = GCN1(A,X);

[0018] Among them, A represents the adjacency matrix of the text co-occurrence graph, X represents the word embedding vector matrix, and h i represents the feature vector corresponding to the i-th word in the text co-occurrence graph, S is the number of words in the vocabulary, and GCN1 represents the text co-occurrence graph convolutional network module;

[0019] Then, the attention mechanism is used to weight the embedding vector of each node of the feature matrix H of the text co-occurrence graph to obtain the weighted feature matrix of the text co-occurrence graph:

[0020] H′=[h′1,…,h′ i ,…,h′ S ],

[0021] where h′ i =tanh(MLP1(h i ))*sigmoid(MLP2(h i )), MLP1 and MLP2 represent multi-layer perceptrons.

[0022] Furthermore, the word embedding vector matrix and the adjacency matrix of the syntactic dependency graph are input into the syntactic dependency graph convolutional network module to obtain a weighted feature matrix of the syntactic dependency graph, including:

[0023] First, input the word embedding vector matrix and the adjacency matrix of the syntactic dependency graph into the syntactic dependency graph convolutional network to obtain the feature matrix of the syntactic dependency graph:

[0024] M=[m1,…,m i ,…,m s ] = GCN2(C,X);

[0025] Among them, C represents the adjacency matrix of the syntactic dependency graph, X represents the word embedding vector matrix, and m i represents the feature vector corresponding to the i-th word in the syntactic dependency graph, S is the number of words in the vocabulary, and GCN2 represents the syntactic dependency graph convolutional network module;

[0026] Then, the attention mechanism is used to weight the embedding vector of each node of the feature matrix M of the syntactic dependency graph to obtain the weighted feature matrix of the syntactic dependency graph:

[0027] M′=[m′1,…,m′ i ,…,m′ S ],

[0028] where m′ i =tanh(MLP3(m i ))*sigmoid(MLP4(m i )), MLP3 and MLP4 represent multi-layer perceptrons.

[0029] Furthermore, the hybrid pooling module is used to perform a pooling operation on the weighted feature matrix of the text co-occurrence graph and the weighted feature matrix of the syntactic dependency graph to obtain a feature vector of the text co-occurrence graph and a feature vector of the syntactic dependency graph, wherein:

[0030] The feature vector f of the text co-occurrence graph G1 =(Maxpooling(H′)+Avgpooling(H′)) / 2; The feature vector f of the syntactic dependency graph G2 =(Maxpooling(M′)+Avgpooling(M′)) / 2, H′ represents the weighted feature matrix of the text co-occurrence graph, M′ represents the weighted feature matrix of the syntactic dependency graph, Maxpooling represents the maximum pooling operation, and Avgpooling represents the average pooling operation.

[0031] Furthermore, the adaptive feature fusion module is used to adaptively fuse the feature vector of the text co-occurrence graph and the feature vector of the syntactic dependency graph, including:

[0032] f G =af G1 +(1-a)f G2 ;

[0033] Among them, a represents the trainable parameter, f G1 The feature vector representing the text co-occurrence graph, f G2 The feature vector representing the syntactic dependency graph, f G Represents fusion features.

[0034] Furthermore, the text co-occurrence graph convolutional network module, the syntactic dependency graph convolutional network module, the mixed pooling module, the adaptive feature fusion module and the text classifier module constitute a graph convolutional network model, and the loss function of the graph convolutional network model is:

[0035]

[0036] in, represents the true label of movie review text i, Y i represents the predicted label of movie review text i, and B represents the number of texts.

[0037] The present application also proposes a film review sentiment classification device based on a graph neural network, comprising a processor and a memory storing a plurality of computer instructions, wherein the computer instructions, when executed by the processor, implement the steps of the film review sentiment classification method based on a graph neural network.

[0038] The present application proposes a method and device for sentiment classification of movie reviews based on graph neural networks, which constructs the text sentences into a graph data structure based on their own unique grammatical information and the window co-occurrence relationship of text words, so that the learned network representation not only contains information about neighboring words, but also contains information about more distant words that are related due to grammatical relationships, thereby improving the accuracy of sentiment classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of the method for sentiment classification of movie reviews based on graph neural networks in this application;

[0040] Figure 2 This is a schematic diagram of the convolutional network model structure for this application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] In one embodiment, Figure 1 As shown, a movie review sentiment classification method based on graph neural network is provided, including:

[0043] Step S1, collect movie review text datasets, preprocess each movie review text to obtain a text vocabulary, represent the words in the text vocabulary with embedding vectors, and obtain a word embedding vector matrix of the movie review text.

[0044] Collect movie review text data sets, the number of texts is B, the data set includes the text content of movie reviews and the sentiment labels corresponding to the texts. Perform preprocessing on the original text of each movie review, including word segmentation, stop word removal, punctuation removal, etc., to obtain the preprocessed text.

[0045] For any movie review text i, get the vocabulary W of the text i , W i The number of words in is S, W i Expressed as Use the open source Chinese Word Vector pre-trained word embedding vector to represent each word in the vocabulary, and further obtain the word embedding vector matrix of text i in, represents the jth word in the vocabulary of text i, D represents the word embedding vector dimension, x ik The embedding vector representing the kth word in the vocabulary of document i.

[0046] Traverse the entire movie review text dataset and obtain the word embedding vector matrix of each text. The embedding vector matrix of B texts can be expressed as X = [X1, X2, …, X B ], the vocabulary of B movie review texts can be expressed as W = [W 1 ,W 2 ,…,W B ].

[0047] Step S2: construct a text co-occurrence graph by taking the words in the text vocabulary as nodes and the co-occurrence relationships between the words as edges, and obtain an adjacency matrix of the text co-occurrence graph.

[0048] In this step, for any movie review text i, the vocabulary W i The words in the text are used as nodes, and the co-occurrence relationship between words is used as the edge to construct the text co-occurrence graph.

[0049] Specifically, the sliding window size is set to m, and the window slides from left to right along the text sequence. The center word of the window is if and In one window, build and The edges between word nodes are Indicates the number of times two nodes co-occur in the sliding window and calculates the weight of the edge connecting the two nodes Get the adjacency matrix of the co-occurrence graph of text i

[0050] Traverse the entire movie review text dataset and construct a text co-occurrence graph for each movie review text. The adjacency matrix of B text co-occurrence graphs is

[0051] Step S3: construct a syntactic dependency graph of the text using words in the text vocabulary as nodes and syntactic relationships as edges, and obtain an adjacency matrix of the syntactic dependency graph.

[0052] This step performs syntactic analysis on any movie review text. The vocabulary W i The words in are used as nodes, and the syntactic relationships are used as edges to construct the syntactic dependency graph of text i. Get the adjacency matrix C i ,in:

[0053]

[0054] or 0 respectively indicates that there is or is no edge between node j and node k in movie review text i; traverse the entire movie review text dataset, build a syntactic dependency graph for each movie review text, and obtain the adjacency matrix of B text syntactic dependency graphs

[0055] Step S4: input the word embedding vector matrix and the adjacency matrix of the text co-occurrence graph into the text co-occurrence graph convolutional network module to obtain the weighted feature matrix of the text co-occurrence graph.

[0056] Construct a graph convolutional network model, such as Figure 2As shown in the figure, the model is divided into five modules: text co-occurrence graph convolutional network module, syntactic dependency graph convolutional network module, hybrid pooling module, adaptive feature fusion module, and text classifier module.

[0057] This step uses the text co-occurrence graph convolutional network module to learn the embedding representation of text words. The adjacency matrix A and feature matrix X of the movie review text co-occurrence graph are input into the text co-occurrence graph convolutional network module GCN1:

[0058] H=[h1,…,h i ,…,h s ]=GCN1(A,X),

[0059] in, Among them, A represents the adjacency matrix of the text co-occurrence graph, X represents the word embedding vector matrix, and h i represents the feature vector corresponding to the i-th word in the text co-occurrence graph, S is the number of words in the vocabulary, and GCN1 represents the text co-occurrence graph convolutional network module;

[0060] Then, the attention mechanism is used to weight the embedding vector of each node of the feature matrix H of the movie review text co-occurrence graph to obtain the weighted feature matrix of the text co-occurrence graph:

[0061] H′=[h′1,…,h′ i ,…,h′ S ],

[0062] where h′ i =tanh(MLP1(h i ))*sigmoid(MLP2(h i )), MLP1 and MLP2 represent multi-layer perceptrons. tanh represents the hyperbolic tangent activation function, and sigmoid represents the sigmoid activation function. Multi-layer perceptron (MLP) is a relatively mature technology in this field and will not be described here.

[0063] Step S5: input the word embedding vector matrix and the adjacency matrix of the syntactic dependency graph into the syntactic dependency graph convolutional network module to obtain a weighted feature matrix of the syntactic dependency graph.

[0064] This step uses the syntactic dependency graph convolutional network module to learn text word embedding, and inputs the movie review syntactic dependency graph adjacency matrix C and feature matrix X into the syntactic dependency graph convolutional network GCN2:

[0065] M=[m1,…,m i ,…,m s ]=GCN2(C,X),

[0066] in, Among them, C represents the adjacency matrix of the syntactic dependency graph, X represents the word embedding vector matrix, and m i represents the feature vector corresponding to the i-th word in the syntactic dependency graph, S is the number of words in the vocabulary, and GCN2 represents the syntactic dependency graph convolutional network module;

[0067] Use the attention mechanism to weight the embedding vector of each node of the feature matrix M of the syntactic dependency graph of the movie review text to obtain the weighted feature matrix of the text syntactic dependency graph

[0068] M′=[m′1,…,m′ i ,…,m′ S ],

[0069] where m′ i =tanh(MLP3(m i ))*sigomid(MLP4(m i )), MLP3 and MLP4 represent multi-layer perceptrons.

[0070] Step S6: Use a hybrid pooling module to perform a pooling operation on the weighted feature matrix of the text co-occurrence graph and the weighted feature matrix of the syntactic dependency graph to obtain a feature vector of the text co-occurrence graph and a feature vector of the syntactic dependency graph.

[0071] This step uses the hybrid pooling module to perform pooling operations on the weighted feature matrix of the text co-occurrence graph and the weighted feature matrix of the syntactic dependency graph to obtain the feature vector of the text co-occurrence graph:

[0072] f G1 =(Maxpooling(H′)+Avgpooling(H′)) / 2,

[0073] And the feature vector of the text syntactic dependency graph:

[0074] f G2 =(Maxpooling(M′)+Avgpooling(M′)) / 2,

[0075] H′ represents the weighted feature matrix of the text co-occurrence graph, M′ represents the weighted feature matrix of the syntactic dependency graph, Maxpooling represents the maximum pooling operation, and Avgpooling represents the average pooling operation.

[0076] Step S7: Adopt an adaptive feature fusion module to adaptively fuse the feature vectors of the text co-occurrence graph and the feature vectors of the syntactic dependency graph.

[0077] This step uses the adaptive feature fusion module to adaptively fuse the feature vectors of the movie review text co-occurrence graph and syntactic dependency graph:

[0078] f G =af G1 +(1-a)f G2 ,

[0079] Where a represents a trainable parameter. G1 The feature vector representing the text co-occurrence graph, f G2 The feature vector representing the syntactic dependency graph, f G Represents fusion features.

[0080] Step S8: input the adaptively fused fusion features into the text classifier module to predict the sentiment classification results of the movie review text.

[0081] This step uses the text classifier module to predict the sentiment classification results of the movie review text:

[0082] Y = softmax(W5f G ),

[0083] Among them, W5 represents the trainable weight parameter.

[0084] In a specific embodiment, the graph convolutional network model of the present application is trained, and the loss function of the graph convolutional network model is as follows:

[0085]

[0086] in, represents the true label of movie review text i, Y i represents the predicted label of movie review text i, and B represents the number of texts.

[0087] During training, the training samples are input into the graph convolutional network model, and the Adam optimizer and back propagation algorithm are used until the loss value no longer decreases or is less than the specified value, and the training is terminated to obtain the final graph convolutional network model. The movie review text that needs to predict the label is executed from step S1 to step S3, and input into the graph convolutional network model. The model output Y is the sentiment classification result.

[0088] In one embodiment, the present application also provides a film review sentiment classification device based on a graph neural network, comprising a processor and a memory storing a plurality of computer instructions, wherein the computer instructions, when executed by the processor, implement the steps of the film review sentiment classification method based on a graph neural network.

[0089] For the specific definition of the film review sentiment classification device based on graph neural network, please refer to the definition of the film review sentiment classification method based on graph neural network above, which will not be repeated here. The above-mentioned film review sentiment classification device based on graph neural network can be implemented in whole or in part through software, hardware and their combination. It can be embedded in or independent of the processor in the computer device in hardware form, or it can be stored in the memory of the computer device in software form, so that the processor can call and execute the above corresponding operations.

[0090] The memory and the processor are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The memory stores a computer program that can be run on the processor, and the processor implements the network topology layout method in the embodiment of the present invention by running the computer program stored in the memory.

[0091] The memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory is used to store a program, and the processor executes the program after receiving an execution instruction.

[0092] The processor may be an integrated circuit chip with data processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0093] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A movie review sentiment classification method based on graph neural network, characterized in that: The film review sentiment classification method based on graph neural network includes: Collect movie review text datasets, preprocess each movie review text to obtain a text vocabulary, represent the words in the text vocabulary with embedding vectors, and obtain the word embedding vector matrix of the movie review text; The words in the text vocabulary are used as nodes, and the co-occurrence relationship between words is used as the edge to construct a text co-occurrence graph, and the adjacency matrix of the text co-occurrence graph is obtained; The words in the text vocabulary are used as nodes and the syntactic relations are used as edges to construct the syntactic dependency graph of the text, and the adjacency matrix of the syntactic dependency graph is obtained; Input the word embedding vector matrix and the adjacency matrix of the text co-occurrence graph into the text co-occurrence graph convolutional network module to obtain the weighted feature matrix of the text co-occurrence graph; Input the word embedding vector matrix and the adjacency matrix of the syntactic dependency graph into the syntactic dependency graph convolutional network module to obtain the weighted feature matrix of the syntactic dependency graph; A hybrid pooling module is used to perform pooling operations on the weighted feature matrix of the text co-occurrence graph and the weighted feature matrix of the syntactic dependency graph to obtain the feature vector of the text co-occurrence graph and the feature vector of the syntactic dependency graph; Adopting the adaptive feature fusion module to adaptively fuse the feature vectors of the text co-occurrence graph and the feature vectors of the syntactic dependency graph; The fused features after adaptive fusion are input into the text classifier module to predict the sentiment classification results of the movie review text; The step of inputting the word embedding vector matrix and the adjacency matrix of the text co-occurrence graph into the text co-occurrence graph convolutional network module to obtain the weighted feature matrix of the text co-occurrence graph includes: First, input the word embedding vector matrix and the adjacency matrix of the text co-occurrence graph into the text co-occurrence graph convolutional network module to obtain the feature matrix of the text co-occurrence graph: H=[h1,…,h i ,…,h s ]=GCN1(A,X); Among them, A represents the adjacency matrix of the text co-occurrence graph, X represents the word embedding vector matrix, and h i represents the feature vector corresponding to the i-th word in the text co-occurrence graph, S is the number of words in the vocabulary, and GCN1 represents the text co-occurrence graph convolutional network module; Then, the attention mechanism is used to weight the embedding vector of each node of the feature matrix H of the text co-occurrence graph to obtain the weighted feature matrix of the text co-occurrence graph: H′=[h′1,…,h′ i ,…,h′ s ], where h′ i =tanh(MLP1(h i ))*sigmoid(MLP2(h i ), MLP1 and MLP2 represent multi-layer perceptrons; The word embedding vector matrix and the adjacency matrix of the syntactic dependency graph are input into the syntactic dependency graph convolutional network module to obtain a weighted feature matrix of the syntactic dependency graph, including: First, input the word embedding vector matrix and the adjacency matrix of the syntactic dependency graph into the syntactic dependency graph convolutional network to obtain the feature matrix of the syntactic dependency graph: M=[m1,…,m i ,…,m s ]=GCN2(C,X); Among them, C represents the adjacency matrix of the syntactic dependency graph, X represents the word embedding vector matrix, and m i represents the feature vector corresponding to the i-th word in the syntactic dependency graph, S is the number of words in the vocabulary, and GCN2 represents the syntactic dependency graph convolutional network module; Then, the attention mechanism is used to weight the embedding vector of each node of the feature matrix M of the syntactic dependency graph to obtain the weighted feature matrix of the syntactic dependency graph: M′=[m′1,…,m′ i ,…,m′ s ], where m′ i =tanh(MLP3(m i ))*sigmoid(MLP4(m i )), MLP3 and MLP4 represent multi-layer perceptrons.

2. The method for sentiment classification of film reviews based on graph neural network according to claim 1 is characterized in that: The hybrid pooling module is used to perform a pooling operation on the weighted feature matrix of the text co-occurrence graph and the weighted feature matrix of the syntactic dependency graph to obtain the feature vector of the text co-occurrence graph and the feature vector of the syntactic dependency graph, wherein: The feature vector f of the text co-occurrence graph G1 =(Maxpooling(H′)+Avgpooling(H′)) / 2; The feature vector f of the syntactic dependency graph G2 =(Maxpooling(M′)+Avgpooling(M′)) / 2, H′ represents the weighted feature matrix of the text co-occurrence graph, M′ represents the weighted feature matrix of the syntactic dependency graph, Maxpooling represents the maximum pooling operation, and Avgpooling represents the average pooling operation.

3. The method for sentiment classification of film reviews based on graph neural network according to claim 1 is characterized in that: The method of using an adaptive feature fusion module to adaptively fuse the feature vector of the text co-occurrence graph and the feature vector of the syntactic dependency graph includes: f G =of G1 +(1-a)f G2 ; Among them, a represents the trainable parameter, f G1 The feature vector representing the text co-occurrence graph, f G2 The feature vector representing the syntactic dependency graph, f G Represents fusion features.

4. The method for sentiment classification of film reviews based on graph neural network according to claim 1 is characterized in that: The text co-occurrence graph convolutional network module, the syntactic dependency graph convolutional network module, the mixed pooling module, the adaptive feature fusion module and the text classifier module constitute a graph convolutional network model, and the loss function of the graph convolutional network model is: in, represents the true label of movie review text i, Y i represents the predicted label of movie review text i, and B represents the number of texts.

5. A film review sentiment classification device based on graph neural network, comprising a processor and a memory storing a plurality of computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method described in any one of claims 1 to 4 are implemented.

Images