Aspect-level sentiment analysis method and system based on gated dilated convolution and graph convolution

Abstract

The present invention relates to an aspect-level sentiment analysis method and system based on gated dilated convolution and graph convolution, and the method comprises: representing the context and aspect words in the sentences of the text to be analyzed by word vectors respectively, and obtaining the embedding of the context and aspect words respectively; inputting the obtained embedding of the context and aspect words into a gated dilated convolutional neural network to obtain the context representation incorporating the aspect information; inputting the context representation incorporating the aspect information into a graph convolutional neural network to obtain the predicted sentiment polarity of the text to be analyzed. The present invention combines the gated dilated convolutional network and the graph convolutional neural network, effectively utilizes the text language information, and at the same time emphasizes the importance of aspect information in aspect-level sentiment analysis, thereby improving the accuracy of the aspect-level sentiment analysis task.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning and natural language processing, and specifically relates to an aspect-level sentiment analysis method and system based on gated dilated convolution and graph convolution. Background Art

[0002] As users' requirements for text sentiment analysis are increasing, sentiment analysis is shifting from a coarse-grained level to a fine-grained level. Aspect-Based Sentiment Analysis (ABSA) aims to identify the sentiment category of each specific aspect in a given text. As one of the important subtasks of sentiment analysis, it can mine users' more specific and deeper emotions for different aspects, thus becoming an important basis for decision-making in multiple fields. It has great application value and has evolved into an emerging research direction for natural language processing tasks.

[0003] In early studies, aspect-level sentiment analysis was treated as a general sentiment analysis task, and classification models were established using traditional machine learning methods and dictionary- and rule-based methods. However, this method requires a lot of preprocessing of the input text, and the model performance depends on manually selected features, so the model has poor generalization. With the improvement of computer processing speed, end-to-end deep learning methods are more commonly used, which achieve better results than traditional methods. Most of the models proposed in the past are based on attention mechanisms and common neural networks, which have made great contributions to how to extract semantic information. However, they ignore the grammatical information of text data, which is also crucial for the discrimination of aspect-level sentiment analysis. Based on the study of graph convolutional networks (GCN), it is found that graph convolution can ensure the encoding of the structure represented by the sentence dependency tree, effectively capture syntactic information and long-distance word dependencies, and use GCN networks in ABSA models to extract text grammatical information. Gated atrous convolution can further enhance the effective information of the data on the basis of convolution, and at the same time integrate aspect information into context information, so as to consider aspect features when encoding context. Aspect item information is also crucial to the performance of ABSA tasks. Summary of the invention

[0004] In order to make full use of the grammatical information in the text and improve the accuracy of aspect-level sentiment analysis, the first aspect of the present invention provides an aspect-level sentiment analysis method based on gated dilated convolution and graph convolution, including: representing the context and aspect words in the sentences of the text to be analyzed with word vectors, respectively, to obtain the embedding of the context and aspect words, respectively; inputting the obtained embedding of the context and aspect words into a gated dilated convolutional neural network to obtain a context representation that incorporates the aspect information; inputting the context representation that incorporates the aspect information into a graph convolutional neural network to obtain the predicted sentiment polarity of the text to be analyzed.

[0005] In some embodiments of the present invention, the step of representing the context and aspect words in the sentence of the text to be analyzed by using word vectors respectively includes:

[0006] Perform multi-dimensional GloVe word embedding on context and aspect words respectively to generate context embedding and aspect word embedding in represents a set of real numbers, m and n represent the context embedding v s and aspect word embedding v t The length, d w Represents the dimension of the word vector.

[0007] In some embodiments of the present invention, the step of inputting the obtained context and aspect word embeddings into a gated atrous convolutional neural network to obtain a context representation incorporating aspect information comprises the following steps:

[0008] Stack three dilated convolutional layers with dilation rates of 1, 2, and 4 to get the context hidden state vector Where τ, m, and n represent the number of nodes in each dilated convolutional layer;

[0009] The state vector H is hidden to the context through a gating mechanism c Incorporate information into the

[0010] In some embodiments of the present invention, the graph convolutional neural network includes an input layer, a graph convolutional layer, a pooling layer and a softmax layer. The input layer is used to receive a contextual representation that incorporates aspect information, and obtain an adjacency matrix based on a sentence dependency tree corresponding to the text; the convolutional layer is used to convert the contextual representation that incorporates aspect information into an aspect vector through an adjacency matrix and graph convolution operations; the pooling layer is used to aggregate and enhance the information of the aspect vector; and the softmax layer is used to predict the probability of sentiment polarity based on the aggregated enhanced information of the aspect vector.

[0011] Furthermore, the graph convolution operation includes: firstly performing a linear transformation on the contextual representation of the input incorporating the aspect information, then performing adjacent summation through an adjacency matrix, and finally performing a nonlinear transformation using a ReLU activation function.

[0012] In the above embodiment, before inputting the context representation incorporating the aspect information into the graph convolutional neural network to obtain the predicted sentiment polarity of the text to be analyzed, the following steps are also included:

[0013] A location-aware transformation is performed on the contextual representation of the incorporated aspect information.

[0014] A second aspect of the present invention provides an aspect-level sentiment analysis system based on gated dilated convolution and graph convolution, comprising:

[0015] An embedding module is used to represent the context and aspect words in the sentences of the text to be analyzed with word vectors, respectively, to obtain the embedding of the context and aspect words;

[0016] The fusion module inputs the obtained context and aspect word embeddings into the gated atrous convolutional neural network to obtain the context representation that incorporates the aspect information;

[0017] The prediction module inputs the context representation incorporating the aspect information into a graph convolutional neural network to obtain the predicted sentiment polarity of the text to be analyzed.

[0018] Furthermore, the embedding module includes a first embedding unit and a second embedding unit, wherein the first embedding unit performs multi-dimensional Glove word embedding on the context of the text to be analyzed to generate context embedding; and the second embedding unit performs multi-dimensional Glove word embedding on the aspect words of the text to be analyzed to generate aspect word embedding.

[0019] The third aspect of the present invention provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the aspect-level sentiment analysis method based on gated hole convolution and graph convolution provided by the first aspect of the present invention.

[0020] A fourth aspect of the present invention provides a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the aspect-level sentiment analysis method based on gated dilated convolution and graph convolution provided in the first aspect of the present invention is implemented.

[0021] The beneficial effects of the present invention are:

[0022] 1. The present invention adopts a gated atrous convolutional network (gated atrous convolutional neural network) to expand the receptive field of convolution and enrich the extraction of local feature information. Based on convolution (convolutional neural network, CNN), the gating mechanism further strengthens the effective information of the data and integrates the aspect information into the context information, so as to consider the aspect features when encoding the context, and effectively improve the aspect-level sentiment analysis results;

[0023] 2. The present invention uses position-aware transformation to introduce the position information of aspect items into the model, which solves the problem of confusion in feature learning of aspect items when different aspect items in a sentence contain the same word, and improves the performance of information retrieval;

[0024] 3. The present invention adopts the graph convolution method to encode the dependency structure represented by the sentence dependency tree, effectively capturing syntactic information and long-distance word dependencies, and further improving the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A basic flow chart of an aspect-level sentiment analysis method based on gated dilated convolution and graph convolution in some embodiments of the present invention;

[0026] Figure 2 A schematic diagram of a specific process of an aspect-level sentiment analysis method based on gated dilated convolution and graph convolution in some embodiments of the present invention;

[0027] Figure 3 Schematic diagram of the principle of a gated atrous convolutional neural network in some embodiments of the present invention;

[0028] Figure 4 Schematic diagram of the structure of an aspect-level sentiment analysis system based on gated dilated convolution and graph convolution in some embodiments of the present invention;

[0029] Figure 5 It is a schematic diagram of the structure of an electronic device in some embodiments of the present invention. DETAILED DESCRIPTION

[0030] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0031] Example 1

[0032] refer to Figure 1 and Figure 2In a first aspect of the present invention, an aspect-level sentiment analysis method based on gated atrous convolution and graph convolution is provided, including: S100. Representing the context and aspect words in the sentences of the text to be analyzed with word vectors respectively, and obtaining embeddings of the context and aspect words respectively; S200. Inputting the obtained embeddings of the context and aspect words into a gated atrous convolutional neural network to obtain a context representation incorporating aspect information; S300. Inputting the context representation incorporating aspect information into a graph convolutional neural network to obtain a predicted sentiment polarity of the text to be analyzed.

[0033] It can be understood that the embedding and context representation of the above context and aspect words are usually represented in the form of vectors or matrices, and aspect words can also be represented as aspect items; context represents one or more related sentences or fragments of the text. Aspect: Aspect is the entity attribute of the object of the review (for example, in a restaurant review, an aspect can be the price, quality, etc. of the food). Aspect is a higher-level concept, and the scale of the aspect concept set is very small compared to the vocabulary contained in the review text. Sentiment Polarity: Sentiment polarity refers to the positive, negative, neutral or conflicting emotions expressed in the review text. Among them, conflict means that the review text expresses both positive and negative emotions to the evaluation object.

[0034] In step S100 of some embodiments of the present invention, the step of representing the context and aspect words in the sentence of the text to be analyzed by using word vectors respectively includes:

[0035] Perform multi-dimensional GloVe word embedding on context and aspect words respectively to generate context embedding and aspect word embedding in represents a set of real numbers, m and n represent the context embedding v s and aspect word embedding v t The length, d w Represents the dimension of the word vector. Specifically, the above steps include:

[0036] S101. Input sentence s=[w1,w2,…,w τ+1 ,…,w τ+m ,…,w n-1 ,wn] as context information, extract the aspect word information of length m ; S102. Use the pre-trained Glove model to represent the context and aspect word set information of the predicted polarity and generate context embedding and aspect word embedding where d w Represents the dimension of the word vector. Optionally, the above Glove model can also use the skip-gram or CBOW model to embed the above word vector.

[0037] refer to Figure 3 In step S200 of some embodiments of the present invention, the step of inputting the obtained context and aspect word embeddings into a gated dilated convolutional neural network to obtain a context representation incorporating aspect information comprises the following steps:

[0038] S201. Stack three dilated convolutional layers with dilation rates of 1, 2, and 4 to obtain the context hidden state vector Where τ, m, and n represent the number of nodes in each dilated convolutional layer;

[0039] S202. Hide the state vector H to the context through a gating mechanism c Incorporate information into the

[0040] It can be understood that the context hidden state vector H c That is, the context representation H c , therefore, the specific calculation process of the gating mechanism includes:

[0041] S2021. The context representation H obtained by the hole convolution encoding c Divided into two equal parts, X and Y, both with latitude d, denoted as H c =[X C Y C ];

[0042] S2022.By Y C The sigmoid activation function on the result is used to obtain the probability g. Let g and 1-g be dot-producted with the context representation X and the aspect word embedding Aspect respectively and add them together to obtain the context representation that incorporates the aspect information. The operations within each convolutional gating unit are as follows:

[0043] H ca =Aspect⊙σ(Y C )+X C ⊙(1-σ(Y C ))

[0044] Where σ is the sigmoid activation function and ⊙ represents the Hadamard product.

[0045] In some embodiments of the present invention, the graph convolutional neural network includes an input layer, a graph convolutional layer, a pooling layer and a softmax layer. The input layer is used to receive a contextual representation that incorporates aspect information, and obtain an adjacency matrix based on a sentence dependency tree corresponding to the text; the convolutional layer is used to convert the contextual representation that incorporates aspect information into an aspect vector through an adjacency matrix and graph convolution operations; the pooling layer is used to aggregate and enhance the information of the aspect vector; and the softmax layer is used to predict the probability of sentiment polarity based on the aggregated enhanced information of the aspect vector.

[0046] Furthermore, the graph convolution operation includes: firstly performing a linear transformation on the contextual representation of the input incorporating the aspect information, then performing adjacent summation through an adjacency matrix, and finally performing a nonlinear transformation using a ReLU activation function.

[0047] Specifically, the above graph convolution operation is also the process of extracting text grammatical information. Therefore, the specific steps of graph convolution to extract text grammatical information are as follows:

[0048] S301. Consider the sentence dependency tree corresponding to the text as a graph G with n nodes, and enumerate the graph to obtain the connection matrix A ij ; Optionally, call the interface of the SpaCy tool to perform dependency analysis on the sentence and generate the corresponding dependency tree in the preprocessing stage, where the nodes represent text words and the edges represent the grammatical dependency relationship between words.

[0049] S302. Construct an L-layer graph convolutional network on the sentence dependency tree. The input of the first layer of graph convolutional network is the hidden layer state vector H of the gated hole convolutional network. 0 =H ca .H 0 and H L They represent the input of the first layer of the graph convolutional network and the output after being updated by L layers of graph convolution. and Represent the state vectors of the i-th word element in the l-th layer and the previous layer respectively;

[0050] S303, in Perform position-aware transformations on graphs before feeding them into successive graph convolutional networks

[0051] Specifically, the function that assigns position weights The specific representation is as follows:

[0052]

[0053]

[0054] in is the position weight of the i-th layer.

[0055] S304: Update the representation of each node through the graph convolution operation to obtain the output of the Lth layer graph convolution The graph convolution process includes: first linearly transforming the input, then performing adjacent summation through the adjacency matrix, and finally using the ReLU activation function for nonlinear transformation. The graph convolution operation of each node is as follows:

[0056]

[0057] Among them, W l represents the weight of the linear transformation, b l is the bias term, and σ is the ReLU activation function. Optionally, average pooling is used to aggregate the information on the enhanced aspect vector, as follows:

[0058]

[0059] where f(·) is the average pooling function.

[0060] Furthermore, the vector obtained after average pooling represents H P The probability representation of different sentiment polarities is obtained through the softmax layer Get the final prediction result. Among them, d p The dimension is the same as that of the sentiment label, and ternary sentiment classification is adopted here.

[0061] In the above embodiment, the model achieves optimal performance when the number of layers is 2, and a two-layer graph convolutional network is selected, that is, L=2.

[0062] Example 2

[0063] refer to Figure 4 In a second aspect of the present invention, an aspect-level sentiment analysis system 1 based on gated dilated convolution and graph convolution is provided, comprising:

[0064] An embedding module 11 is used to represent the context and aspect words in the sentences of the text to be analyzed by word vectors, respectively, to obtain embeddings of the context and aspect words respectively;

[0065] The fusion module 12 inputs the obtained context and aspect word embeddings into the gated atrous convolutional neural network to obtain the context representation incorporating the aspect information;

[0066] The prediction module 13 inputs the context representation incorporating the aspect information into a graph convolutional neural network to obtain a predicted sentiment polarity of the text to be analyzed.

[0067] Furthermore, the embedding module 11 includes a first embedding unit and a second embedding unit. The first embedding unit performs multi-dimensional Glove word embedding on the context of the text to be analyzed to generate context embedding; the second embedding unit performs multi-dimensional Glove word embedding on the aspect words of the text to be analyzed to generate aspect word embedding.

[0068] Example 3

[0069] refer to Figure 5 According to a third aspect of the present invention, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided by the first aspect of the present invention.

[0070] The electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0071] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 5 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0072] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, the above functions defined in the method of the embodiment of the present disclosure are executed. It should be noted that the computer-readable medium described in the embodiment of the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In an embodiment of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In an embodiment of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the foregoing.

[0073] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more computer programs. When the one or more programs are executed by the electronic device, the electronic device:

[0074] Computer program code for performing the operations of embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, Python, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0075] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0076] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. Aspect-level sentiment analysis method based on gated dilated convolution and graph convolution, characterized by: include: The context and aspect words in the sentences of the text to be analyzed are represented by word vectors respectively, and the embedding of the context and aspect words is obtained respectively; The obtained embeddings of context and aspect words are input into the gated dilated convolutional neural network to obtain the context representation incorporating aspect information: stack three dilated convolutional layers with dilation rates of 1, 2, and 4 to obtain the context hidden state vector ,in τ, m and n Respectively represent the number of nodes in each hole convolution layer; the context hidden state vector is added to the context through the gating mechanism. Information on integration; The context representation incorporating the aspect information is input into a graph convolutional neural network to obtain the predicted sentiment polarity of the text to be analyzed.

2. The aspect-level sentiment analysis method based on gated dilated convolution and graph convolution according to claim 1, characterized in that: The step of representing the context and aspect words in the sentence of the text to be analyzed by using word vectors respectively includes: Perform multi-dimensional GloVe word embedding on context and aspect words respectively to generate context embedding and aspect word embedding ,in represents the set of real numbers, m , n Represents context embedding and aspect word embedding Length, Represents the dimension of the word vector.

3. The aspect-level sentiment analysis method based on gated dilated convolution and graph convolution according to claim 1, characterized in that: The graph convolutional neural network includes an input layer, a graph convolution layer, a pooling layer and a softmax layer. The input layer is used to receive the context representation incorporating the aspect information and obtain an adjacency matrix according to the sentence dependency tree corresponding to the text; The convolution layer is used to convert the context representation incorporating aspect information into an aspect vector through an adjacency matrix and a graph convolution operation; The pooling layer is used to aggregate the information of the enhanced aspect vector; The softmax layer is used to predict the probability of sentiment polarity based on the information of the aggregated enhanced aspect vector.

4. The aspect-level sentiment analysis method based on gated dilated convolution and graph convolution according to claim 3, characterized in that: The graph convolution operation includes: firstly, linearly transforming the context representation of the input into the aspect information, then performing adjacent summation through the adjacency matrix, and finally using The activation function performs non-linear transformation.

5. The aspect-level sentiment analysis method based on gated dilated convolution and graph convolution according to any one of claims 1 to 4, characterized in that: Before inputting the context representation incorporating the aspect information into the graph convolutional neural network to obtain the predicted sentiment polarity of the text to be analyzed, the method further includes: A location-aware transformation is performed on the contextual representation of the incorporated aspect information.

6. An aspect-level sentiment analysis system based on gated dilated convolution and graph convolution, characterized in that: include: An embedding module is used to represent the context and aspect words in the sentences of the text to be analyzed with word vectors, respectively, to obtain the embedding of the context and aspect words; The fusion module is used to input the obtained context and aspect word embeddings into the gated dilated convolutional neural network to obtain the context representation that incorporates the aspect information: stack three dilated convolutional layers with dilation rates of 1, 2, and 4 to obtain the context hidden state vector ,in τ, m and n Respectively represent the number of nodes in each hole convolution layer; the context hidden state vector is added to the context through the gating mechanism. Information on integration; The prediction module is used to input the context representation incorporating the aspect information into a graph convolutional neural network to obtain the predicted sentiment polarity of the text to be analyzed.

7. The aspect-level sentiment analysis system based on gated dilated convolution and graph convolution according to claim 6, characterized in that: The embedded module includes a first embedded unit and a second embedded unit. The first embedding unit performs multi-dimensional Glove word embedding on the context of the text to be analyzed to generate context embedding; The second embedding unit performs multi-dimensional GloVe word embedding on the aspect words of the text to be analyzed to generate aspect word embedding.

8. An electronic device, comprising: one or more processors; A storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the aspect-level sentiment analysis method based on gated hole convolution and graph convolution as described in any one of claims 1 to 5.

9. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the aspect-level sentiment analysis method based on gated hole convolution and graph convolution is implemented as described in any one of claims 1 to 5.

Images