An aspect-level sentiment classification method based on a graph attention network

Abstract

The application belongs to the technical field of natural language processing, and particularly relates to an aspect-level sentiment classification method based on a graph attention network, which comprises the following steps: obtaining word embedding representation of context text in which an aspect word is located; dynamically adjusting the weight of a context word according to the relative position of the context word and the aspect word, and obtaining context semantic features; aggregating syntactic information through an improved graph attention network to obtain syntactic features of the text; using a deep cross network to fuse the syntactic features of the text and the context semantic features to obtain final feature representation; and performing sentiment prediction on the final feature representation through a full connection layer to obtain the sentiment polarity distribution of the aspect word in the text. The application solves the problem of feature information loss that may occur in a multilayer network of the graph attention network, and considers the position information of the context word when extracting semantic features, so that the syntactic features and the context semantic features are fully fused, thereby improving the accuracy of aspect-level sentiment classification.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, specifically relating to an aspect-level sentiment classification method based on graph attention networks. Background Technology

[0002] With the widespread adoption of the internet, e-commerce, social media, and other application industries have developed rapidly, and the global internet user base has continued to expand. More and more people are shopping through e-commerce platforms and leaving product reviews, sharing their opinions, thoughts, and attitudes on social media platforms. This has resulted in a massive amount of valuable user comments on topics such as people, events, and products, containing users' emotional inclinations. This vast amount of text data typically contains enormous commercial value, but how to extract effective information from it is a crucial question. Taking online shopping as an example, the sheer volume of reviews makes it difficult for users to quickly locate specific aspects of a product they are interested in. Simply browsing online reviews to obtain a comprehensive evaluation of a product is unrealistic. Traditional sentiment classification only identifies the overall sentiment of a review, neglecting specific aspects and attributes of the product. This coarse-grained sentiment recognition not only leads to insufficient information extraction but also fails to identify the specific emotional inclinations of the product that users are interested in. Therefore, it is necessary to establish a system for aspect-level sentiment classification of text. With the rapid development of deep learning algorithms, utilizing relevant technologies in the field of natural language processing for aspect-level sentiment classification of text can significantly improve and enhance the acquisition of effective information from text.

[0003] One challenge in aspect-level sentiment classification lies in the loss of feature information that can occur when using graph attention networks in multi-layered networks. Previous methods have largely failed to adequately address how to ensure the effective propagation of feature information. A second challenge is the effective integration of textual syntactic features and contextual semantic features. A comment contains not only semantic features related to the context but also syntactic features containing syntactic relationships. Both are crucial for aspect-level sentiment classification. Previous studies have mostly considered only semantic features or failed to effectively integrate the two, resulting in incomplete and inaccurate output features. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes an aspect-level sentiment classification method based on graph attention networks, comprising:

[0005] S1: Obtain the word embedding representation of the context text where the aspect words are located, and get the word vector corresponding to the sentence;

[0006] S2: Dynamically adjust the weights of context words in the word vectors based on their relative positions with context words and aspect words to obtain contextual semantic features;

[0007] S3: Aggregate syntactic information and obtain the syntactic features of the text by using an improved graph attention network;

[0008] S4: Use a deep cross-network to fully integrate text syntactic features and contextual semantic features to obtain the final feature representation;

[0009] S5: The final feature representation is passed through a fully connected layer and a softmax function to perform sentiment prediction, thereby obtaining the sentiment polarity distribution of words in that aspect of the text.

[0010] The beneficial effects of this invention are:

[0011] 1. This invention uses a residual network structure to compensate for the problem of feature information loss that occurs in graph attention networks under multi-layer networks;

[0012] 2. This invention uses a deep cross-network to effectively fuse contextual semantic information and syntactic features to obtain the final representation;

[0013] 3. The method of the present invention can achieve better sentiment classification results for specific aspect words in the text, and improve the accuracy of aspect-level sentiment classification. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the overall process of the aspect-level sentiment classification method based on graph attention networks of this invention.

[0015] Figure 2 This is a diagram of the improved graph attention network structure of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] An aspect-level sentiment classification method based on graph attention networks, such as Figure 1 As shown, the method includes the following steps:

[0018] S1: Obtain the word embedding representation of the context text where the aspect words are located, and get the word vector corresponding to the sentence;

[0019] S2: Dynamically adjust the weights of context words in the word vectors based on their relative positions with context words and aspect words to obtain contextual semantic features;

[0020] S3: Aggregate syntactic information and obtain the syntactic features of the text by using an improved graph attention network;

[0021] S4: Use a deep cross-network to fully integrate text syntactic features and contextual semantic features to obtain the final feature representation;

[0022] S5: The final feature representation is passed through a fully connected layer and a softmax function to perform sentiment prediction, thereby obtaining the sentiment polarity distribution of words in that aspect of the text.

[0023] The process of obtaining the word embedding representation of the context text in which the aspect term is located includes:

[0024] Given a sentence s containing n words, s = {w1, w2, ..., wn} t+1 ,...,w t+m ,...,w n}, and aspect words of length m, a = {w t+1 ,...,w t+m In other words, aspect word 'a' is part of sentence 's'. Using a BERT pre-trained model to model the context, to facilitate BERT model training and fine-tuning, the text sequence is reconstructed into the form "[CLS]+s+[SEP]+a+[SEP]". The feature vector output by the BERT model is represented as {e0,e1,...,e...}. n-1 ,e n ,e n+1 ,e t+1 ,...,e t+m ,e n+2+m}, where the feature vector of sentence s is h. s ={e1,e2,...,e n The feature vector of aspect word a is h. a ={e t+1 ,e t+2 ,...,e t+m},remember n represents the length of the text sequence, and d represents the word embedding dimension. Here, BERT stands for Transformer-based bidirectional encoding representation.

[0025] In aspect-level sentiment classification tasks, the contribution of each word in the context to the sentiment of the aspect word varies. Words that have a greater impact on the sentiment of the aspect word are usually located in a local region centered on the aspect word, while words farther away from the aspect word have a relatively smaller impact. Therefore, when extracting semantic features, we should pay more attention to the context words within the local region and reduce the influence of noise words from outside the domain.

[0026] The specific steps for obtaining contextual semantic features are as follows:

[0027] S21: Calculate the semantic relative distance between context words and aspect words. Let D be the semantic relative distance between the t-th word in the context and the aspect word. t The calculation formula is:

[0028]

[0029] Among them, D t P represents the semantic relative distance between the t-th word and the aspect term in the context. t P represents the position of the t-th word in the context. asp The term "len" indicates the central position of the aspect term, "len" indicates the length of the aspect term, and "n" indicates the sentence length. and These represent rounding up and rounding down when the result is not an integer, respectively.

[0030] S22: Obtain contextual semantic features. Let γ be the threshold of the local region centered on the aspect word. When D... t If the value is less than or equal to γ, the word is considered to belong to the local region; otherwise, it belongs to the outside region. For words within the region, we consider them all important, and their weight coefficient is 1. For words outside the region, the weight coefficient varies with D. t The weighting coefficients decrease dynamically with increasing weights; the specific formula is as follows:

[0031]

[0032] W = {w1, w2, ..., w n}

[0033] h ex =h s ·W

[0034] Where E is a unit column vector, w t Let w represent the adjusted weight vector of the t-th word, and W represent the weight matrix of the text. t Let γ represent the adjusted weight vector of the t-th word, γ represent the threshold of the local region centered on the aspect word, and h represent the weight vector of the t-th word. ex This represents the output after the feature extraction layer, i.e., the contextual semantic features.

[0035] The specific steps for obtaining syntactic features include:

[0036] S31: Perform syntactic parsing on the text to generate the corresponding syntactic dependency tree, and interpret it using undirected and directed graph structures to obtain an undirected graph adjacency matrix A containing syntactic relation information. undg And the adjacency matrix A of the directed graph dg ;

[0037] S32: Connect the adjacency matrix A of the undirected graph. undg And the adjacency matrix A of the directed graph dg The inputs are fed into the improved graph attention network (ImGAT) to obtain the undirected graph feature representation. and directed graph feature representation

[0038] S33: The ImGAT network includes: Graph Attention Network (GAT), feedforward neural network, gating mechanism, and residual network, such as Figure 2 As shown, the specific calculation process is as follows:

[0039] The update process of the GAT network is as follows:

[0040]

[0041]

[0042]

[0043] Where K represents the number of attention heads, and N(i) represents the neighboring nodes of node i. W represents the attention weights calculated using the k-th attention head after passing through the l-th layer of GAT. lk This represents the weight parameters of the k-th attention head in the l-th layer model. d represents the features of two nodes i and j in the l-th layer. h The input dimension of the node is represented by T, where T represents the matrix transpose operation, and softmax and LeakyReLU represent the activation functions.

[0044] The syntactic hidden feature information obtained after passing through a single GAT network layer is normalized using a feedforward neural network, as shown in the following formula:

[0045]

[0046] Where W1 and W2 represent weight parameters, and b1 and b2 represent bias parameters. This represents the syntactic hidden features output after passing through a feedforward neural network.

[0047] After passing through the feedforward neural network, a gating function is used to initially fuse semantic information and syntactic hidden feature information, as shown in the following formula:

[0048]

[0049] Where h ex This represents the feature representation output after the feature extraction layer, where * indicates the element-wise multiplication operation, and a t Represents the gate function. This indicates the features obtained after initial fusion through the gating mechanism.

[0050] Finally, a residual network is added to ensure that feature information can be effectively transferred. The formula is as follows:

[0051]

[0052] After passing through multiple layers of the ImGAT network, a deep feature representation containing syntactic features is obtained. This is because the features of any node i in the undirected and directed graphs are obtained at the (l+1)th layer of the GAT. They are different, therefore the final feature representations containing syntactic features obtained after the above process are different. They are also different, and are defined separately. and The sentences composed of nodes are the feature vectors of an undirected graph. and directed graph eigenvectors

[0053] Deep cross-networks are used to fuse feature information to obtain the final feature representation, which consists of a set of cross-networks and a set of fully connected feedforward neural networks. The cross-network propagation process is as follows:

[0054]

[0055] Where w l ,b l These represent the weight parameters and bias parameters, respectively. It is obtained by concatenating two sets of feature vectors. This method uses the feature vectors of an undirected graph. and context feature vector h ex The formula for splicing and merging is as follows:

[0056]

[0057] in, This represents the eigenvectors of an undirected graph. and context feature vector h ex The feature vectors are spliced ​​and fused. This indicates a splicing and merging operation. This represents the fused undirected graph feature vector. and context feature vector h ex The feature vector w used for cross-network propagation l ,b l Let represent the weight parameters and bias parameters of the deep cross-multiplication network, respectively, and T denote the matrix transpose operation. This represents the feature vector output after passing through the l-th layer of the cross-network.

[0058] Directed graph feature vectors that did not participate in the fusion Propagation is performed forward through a fully connected feedforward neural network, as shown in the following formula:

[0059]

[0060] Among them, f fc This represents the directed graph eigenvectors that did not participate in the fusion. The feature vector, w, propagates forward through a fully connected feedforward neural network. fc ,b fc These represent the weight parameters and bias parameters of a fully connected feedforward neural network, respectively, and relu() represents the activation function.

[0061] After the above propagation process, the output vectors from the two networks are concatenated to obtain the final representation h. dcn The formula is as follows.

[0062]

[0063] Among them, h dcn This represents the final feature representation. This represents the fused undirected graph feature vector. and context feature vector h ex The feature vector f used for cross-network propagation fc This represents the directed graph eigenvectors that did not participate in the fusion. The feature vector propagated forward through a fully connected feedforward neural network This indicates a splicing and merging operation.

[0064] The final representation h obtained dcn The input is fed into a fully connected layer, and the softmax function is used for classification. The output predicts the sentiment polarity of the relevant words, as shown in the following formula:

[0065] y = softmax(W dcn h dcn +b dcn )

[0066] Where y represents the predicted sentiment polarity distribution, W dcn ,b dcnh represents the weight and bias parameters of the fully connected layer, respectively. dcn This represents the final feature representation.

[0067] The syntactic dependency tree is interpreted using undirected and directed graph structures to fully acquire the syntactic information of the text. Then, the syntactic information is aggregated through an improved graph attention network, which enables the feature information to be effectively propagated in the multi-layer network. At the same time, the semantic feature representation of the text is generated by using a BERT pre-trained model and a feature extraction layer. Finally, the semantic features and syntactic relations are fused through a deep cross network to obtain the final representation, which is then input into a fully connected layer and a softmax function to obtain an aspect-level sentiment classification model.

[0068] To train the model, input the text to be recognized into the trained model, use cross-entropy as the loss function, and train using standard gradient descent, as shown in the following formula:

[0069]

[0070] Where N represents the total number of training samples, y i This represents the true sentiment polarity of the i-th sample. Let λ represent the predicted sentiment polarity of the i-th sample, λ be the L2 regularization coefficient, and θ represent the set of trainable parameters in the model.

[0071] The Adam optimizer is used to train and optimize an aspect-level sentiment classification model based on graph attention networks. The text to be classified is input into the model to obtain the sentiment classification results of specific aspect words contained in the text; Adam represents the adaptive moment estimate.

[0072] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An aspect-level sentiment classification method based on graph attention networks, characterized in that, include: S1: Obtain the word embedding representation of the context text where the aspect words are located, and get the word vector corresponding to the sentence; S2: Dynamically adjust the weights of context words in the word vectors based on their relative positions with context words and aspect words to obtain contextual semantic features; Obtain contextual semantic features, including: in, This represents the contextual semantic features after the feature extraction layer. This represents the feature vector of sentence S. The weight matrix representing the text. , This represents the adjusted weight vector of the t-th word. E represents a unit column vector. The threshold representing a local region centered on the aspect term. This represents the semantic relative distance between the t-th word and the aspect term in the context. , This indicates the position of the t-th word in the context. Indicates the central position of the aspect word, The length of the aspect term, Indicates sentence length. and These represent rounding up and rounding down when the calculation result is not an integer, respectively. S3: Aggregate syntactic information and obtain the syntactic features of the text by using an improved graph attention network; Obtain the syntactic features of the text, including: S31: Perform syntactic parsing on the text to generate the corresponding syntactic dependency tree. Through the syntactic dependency tree, the input text information is transformed into a syntactic graph structure containing N nodes, which can be either an undirected graph structure or a directed graph structure. Each node in the graph represents a word in the sentence, and the edges represent the syntactic dependencies between each word. The neighboring nodes of node i can be represented as N(i). S32: Input the undirected graph structure and the directed graph structure of the text information into the improved graph attention network ImGAT respectively. The improved graph attention network (ImGAT) includes: graph attention network layer, feedforward neural network layer, gating mechanism layer and residual network layer. Syntactic hidden feature representations of undirected and directed graph structures are extracted through graph attention network layers. The syntactic hidden feature representations of undirected and directed graph structures are normalized through feedforward neural network layers. The semantic information and the normalized syntactic hidden feature representations of undirected and directed graph structures are initially fused through the gating function of the gating mechanism layer. Finally, the feature vectors of undirected and directed graph structures containing syntactic features are output through residual network layers. S4: Use a deep cross-network to fully integrate text syntactic features and contextual semantic features to obtain the final feature representation; By using a deep cross-network to fully integrate text syntactic features and contextual semantic features, the final feature representation is obtained, including: S41: Transform the eigenvectors of the undirected graph and context feature vectors The feature vectors of the merged undirected graph are then concatenated and fused. and context feature vectors Input a deep cross-network for cross-network propagation; S42: The directed graph feature vectors that were not involved in the fusion Propagate forward through a fully connected feedforward neural network; S43: Concatenate the two sets of vectors after propagation to obtain the final feature representation; The two sets of vectors after propagation are concatenated to obtain the final feature representation, including: in, This represents the final feature representation. This represents the fused undirected graph feature vector. and context feature vectors Feature vectors used for cross-network propagation This represents the directed graph eigenvectors that did not participate in the fusion. The feature vector propagated forward through a fully connected feedforward neural network This indicates a splicing and merging operation; S5: The final feature representation is passed through a fully connected layer and a softmax function to perform sentiment prediction, thereby obtaining the sentiment polarity distribution of words in that aspect of the text.

2. The aspect-level sentiment classification method based on graph attention networks according to claim 1, characterized in that, Obtain the word embedding representation of the context text containing the aspect term, including: Given a sentence S containing n words, Extract aspect terms of length m from sentence S. Reconstruct the text sequence as "[CLS]+S+[SEP]+ The format "+[SEP]" is used to model the context using a BERT pre-trained model, and the feature vector output by the BERT model is represented as follows: The feature vector of sentence S is , aspect The feature vector is ,remember , where n represents the length of the text sequence and d represents the word embedding dimension.

3. The aspect-level sentiment classification method based on graph attention networks according to claim 1, characterized in that, eigenvectors of an undirected graph and context feature vectors The feature vectors of the merged undirected graph are then concatenated and fused. and context feature vectors Cross-network propagation includes: splicing and fusion process: Cross-network propagation process: in, This represents the eigenvectors of an undirected graph. and context feature vectors The feature vectors are spliced ​​and fused. This indicates a splicing and merging operation. This represents the fused undirected graph feature vector. and context feature vectors Feature vectors used for cross-network propagation , These represent the weight parameters and bias parameters of the deep cross-network, respectively. This represents the matrix transpose operation. This represents the feature vector output after passing through the l-th layer of the cross-network.

4. The aspect-level sentiment classification method based on graph attention networks according to claim 1, characterized in that, Directed graph feature vectors that were not involved in the fusion Forward propagation via a fully connected feedforward neural network includes: in, This represents the directed graph eigenvectors that did not participate in the fusion. The feature vector propagated forward through a fully connected feedforward neural network , These represent the weight parameters and bias parameters of a fully connected feedforward neural network, respectively. This represents the activation function.

5. The aspect-level sentiment classification method based on graph attention networks according to claim 1, characterized in that, Sentiment prediction is performed using the final feature representation through a fully connected layer and a softmax function, including: in, This indicates a prediction of the distribution of sentiment polarity. , These represent the weight parameters and bias parameters of the fully connected layer, respectively. This represents the final feature representation.

Images