A sentiment analysis method based on a syntax and morphology enhanced graph attention network model
Patent Information
- Application Number
- CN202411071411.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2026-11-03
- Estimated Expiration
- 2044-08-06
AI Technical Summary
Existing aspect-level sentiment analysis methods have limitations in handling long-distance dependencies and multi-dimensional information, making it difficult to fully utilize syntactic structure and lexical information, resulting in insufficient accuracy in sentiment polarity classification.
A graph attention network model based on syntactic and lexical enhancement is adopted. By introducing syntactic distance and part-of-speech weights and combining them with a multi-head attention mechanism, the model's ability to extract features related to aspect words is enhanced, and the syntactic and semantic relationships between words are explicitly expressed.
It improves the accuracy of sentiment polarity classification, better handles complex contexts and long-distance dependencies, and significantly enhances the model's performance on multiple benchmark datasets.
Smart Images
Figure CN119003774B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to aspect-level sentiment analysis methods, specifically, to the design of a sentiment analysis method based on a graph attention network model with syntactic and lexical enhancements. Background Technology
[0002] With the development of natural language processing technology, aspect-based sentiment analysis (ABSA), as an important branch of fine-grained sentiment analysis tasks, has received widespread attention. Its purpose is to predict the sentiment polarity of specific aspect words in a sentence, including positive, negative, and neutral. For example, in the sentence "The price is reasonable although the service is poor," the aspect words "price" and "service" need to have their corresponding sentiments "reasonable" (positive) and "poor" (negative), respectively.
[0003] Early aspect-level sentiment analysis methods primarily relied on traditional machine learning algorithms, such as Support Vector Machines (SVM) and Naive Bayes, which were effective in determining the sentiment polarity of entire sentences. However, these algorithms often struggled to utilize contextual details for more specific analysis, thus limiting their ability to implement aspect-level sentiment analysis.
[0004] With the rapid development of deep learning technology, aspect-level sentiment analysis methods have been significantly improved. For example, convolutional neural networks (CNNs) and recurrent neural networks (RNNs), including long short-term memory networks (LSTMs) and gated recurrent units (GRUs), have been applied to address this problem. These models can significantly improve the performance of aspect-level sentiment analysis by learning complex word embeddings and contextual interactions. Furthermore, the introduction of attention mechanisms further enhances the model's grasp of relevant lexical relationships in context, although it is still affected by sentence noise.
[0005] To address these challenges, researchers have begun employing Graph Convolutional Networks (GCNs). Zhang et al. designed a GCN method based on dependency parse trees, which uses graph convolution propagation mechanisms to pass the representations of context words to the target aspect word by connecting the boundaries between words. Sun et al. proposed a Dependency Tree Convolutional Model (CDT), which uses bidirectional long short-term memory (Bi-LSTM) to learn sentence feature representations and further enhances the embeddings through GCN. Qian et al. proposed hierarchical syntactic graphs and hierarchical lexical graphs, and designed a two-layer interactive graph convolutional network (BiGCN) to fully integrate hierarchical syntactic and lexical graphs.
[0006] However, these models typically assign the same weight to every edge in the graph, failing to adequately consider the crucial role of different words in sentiment analysis. To address this issue, Yuan et al. proposed a graph attention network model (G-ATT) incorporating an adaptive attention mechanism. By assigning unequal weights to different edges in the graph, the model can dynamically learn the importance of edges, thereby improving its ability to capture key sentiment information. Nevertheless, G-ATT still faces some challenges in handling long-distance dependencies, demonstrating potential limitations in multi-dimensional information processing. Summary of the Invention
[0007] In view of the problems and shortcomings of the existing technology in aspect-level sentiment analysis tasks, the present invention provides a sentiment analysis method based on a syntactic-lexical enhanced graph attention network (SLGAT) model. This method comprehensively considers the syntactic structure and lexical information of the text, improves the sentiment polarity classification, and enhances the model's ability to extract aspect-related features by introducing syntactic distance and part-of-speech weights, making it more suitable for the field of sentiment analysis.
[0008] To solve the above problems, the present invention adopts the following technical solution:
[0009] This invention is a sentiment analysis method based on a graph attention network model with syntactic and lexical enhancements. Its key features are as follows:
[0010] A. Text representation:
[0011] The comment text sentence is divided into left-side and right-side text, and a set of aspect terms is embedded into them to form the input sequence of the BERT pre-trained model. The hidden layer state is generated by the BERT model as the aggregate representation of the entire sentence.
[0012] B. Syntactic and lexical enhanced graph attention networks:
[0013] The shortest dependency syntactic path length and syntactic distance vector between each word and aspect term in the text are transformed through a linear layer and processed using a self-attention mechanism. Finally, they are concatenated with the BERT embedding on the feature dimension.
[0014] Simultaneously considering two dimensions—syntactic distance and part-of-speech category—syntactic weights and part-of-speech weights are generated. After normalization, the final weight matrix is generated through multiplication and combination, explicitly expressing the syntactic and semantic relationships between words.
[0015] Multi-head attention mechanism is used to extract multi-dimensional feature information, and the mask vector and input features are combined for weighted summation to focus on the text part related to aspect words.
[0016] The weighted multi-head attention representation is input into a linear layer, and the linear output is converted into a probability distribution through a softmax function, which then classifies the sentiment polarity.
[0017] The preferred technical solution of the sentiment analysis method based on a graph attention network model with syntactic and lexical enhancement described in this invention is as follows: the text representation generation in step (A) comprises the following specific steps:
[0018] A1. The model first divides the comment text into left-side text S. L and the text S on the right R The term set A is embedded between the two. Then, the input sequence of the BERT model is constructed according to the following structure: X = [CLS] + S L +A+S R +[SEP]+A+[SEP].
[0019] A2. Each element in the sequence is converted into a BERT index by a tokenizer, forming the final BERT input sequence. The first element in the hidden layer state generated by the BERT model is ultimately used as the aggregate representation H = [h1, h2, h3, ..., h...]. n ].
[0020] The preferred technical solution of the event prediction method incorporating a hierarchical attention mechanism described in this invention is: the syntactic and lexical enhanced graph attention network described in step (A), the specific steps of which are as follows:
[0021] B1. Using the open-source natural language processing library SpaCy, calculate the shortest dependency syntactic path length and syntactic distance vector between each word in the text and all words within the aspect phrase. Transform the syntactic distance vector through a linear layer, process it using a self-attention mechanism, and finally concatenate it with the BERT embedding along the feature dimension.
[0022]
[0023] d'=wd(2)
[0024]
[0025] E=[H||SA(d')] (4)
[0026] B2. Based on the two dimensions of syntactic distance and part-of-speech category, generate syntactic weights and part-of-speech weights. The normalized weights are then combined through multiplication to obtain the final weight matrix. Where d... i `seq_len` represents the syntactic distance between the i-th word and the aspect word, while `seq_len` represents the feature dimension size of the sentence. It is the calculated syntactic weight. Let A represent the part-of-speech weight of the i-th word, with a threshold of 0.55. m Based on w norm The mask matrix generated by the weight vector.
[0027]
[0028]
[0029] B3. A weighted multi-head attention mechanism is employed to extract multi-dimensional feature information. This is combined with a mask vector and input features, and a weighted sum is performed, focusing on the text portion related to aspect words. The weighted multi-head attention representation is input into a linear layer, and the linear output is transformed into a probability distribution using a softmax function, classifying the output as sentiment polarity. A weight matrix weights∈R is defined. n×n weights i,j This represents the weight between the i-th word and the j-th word. The input data x is transformed linearly to obtain the query matrix Q, the key matrix K, and the value matrix V. and It is a learnable parameter matrix, where i represents the index of the attention head, and d... k W represents the dimension of the key matrix. O It is another learnable parameter matrix.
[0030] Q i =XW i Q (11)
[0031] K i =XW i K (12)
[0032] V i =XW i V (13)
[0033]
[0034] aspect = X·A m (15)
[0035] scores=linear(aspect)+Attention (16)
[0036] scores=softmax(scores·weights) (17)
[0037] Output = scores·V (18)
[0038] M=Concat(head1,head2,...,head h W O (19)
[0039] Compared with the prior art, the present invention has the following beneficial effects:
[0040] By combining syntactic distance and part-of-speech information, the model's ability to extract aspect-related features is enhanced, improving the accuracy of sentiment polarity classification. By dynamically adjusting the weight matrix, the syntactic and semantic relationships between words are explicitly expressed, enabling the model to better handle complex contexts and long-distance dependencies. The proposed SLGAT model outperforms existing methods on multiple benchmark datasets, validating its effectiveness and application value in sentiment analysis tasks.
[0041] This invention comprehensively considers the syntactic structure and lexical information of text, and improves sentiment polarity classification. By introducing syntactic distance and part-of-speech weights, it enhances the model's ability to extract aspect-related features, making it more suitable for the field of sentiment analysis. Experiments on multiple benchmark datasets verify the effectiveness and application value of the proposed SLGAT model.
[0042] To overcome these problems, this paper proposes a Syntactic and Lexical Enhanced Graph Attention Network (SLGAT) model. This model fully mines and utilizes syntactic structure and lexical information, designing a graph attention network model that can effectively extract aspect-word related feature representations and uses a weighted edge enhancement matrix to represent special dependency relations. Through a weighted multi-head attention mechanism, it integrates aspect-word related feature representations and weighted edge embedding information, focusing on special syntactic dependencies.
[0043] Experimental results on multiple benchmark datasets show that the proposed SLGAT model significantly outperforms existing methods on aspect-level sentiment classification tasks, validating its effectiveness and robustness. Attached Figure Description
[0044] Figure 1 This is a structural diagram of the model framework of the present invention;
[0045] Figure 2 This is a dependency tree diagram illustrating the example sentences of this invention;
[0046] Figure 3 This is a schematic diagram of the architecture of the multi-head attention mechanism of the present invention;
[0047] Figure 4 The figures show the ablation experiment results for each dataset in this invention.
[0048] Figure 5This is an experiment comparing the number of GAT layers in various datasets of this invention; Detailed Implementation
[0049] The implementation process of the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.
[0050] Reference Figure 1 The present invention provides aspect-level sentiment analysis using a syntactic-lexical enhanced graph attention network, the method comprising the following steps:
[0051] Step 101: The model first divides the comment text sentence into left-side text S. L and the text S on the right R The term set A is embedded between the two. Then, the input sequence for the BERT model is constructed as follows: X = [CLS] + S L +A+S R +[SEP]+A+[SEP]
[0052] Reference Figure 2 The specific steps are as follows:
[0053] Step 201: Calculate the shortest dependency syntactic path length and syntactic distance vector between each word in the text and all words within the aspect phrase, using the following formula:
[0054]
[0055] The shortest dependency syntax path length is d(s) i ,s j Let s represent the lengths and take the maximum value of these lengths. i It refers to any word in the sentence other than the categorical noun, s j "This is a type of phrase";
[0056] Step 202: Transform the syntactic distance vector d through a linear layer, then use a self-attention mechanism to process the transformed syntactic distance vector, and finally concatenate the BERT embedding and the self-attention output along the feature dimension.
[0057] d'=wd (2)
[0058]
[0059] X=[H||SA(d')] (4)
[0060] Where w represents the linear layer, d k It represents the dimension of the key, SA(d') is the self-attention output, and "||" indicates the concatenation operation. The concatenation process yields the final text representation E = {e1}. l e2 l e3l ...,e n l}, where l represents the output of the last layer, resulting in an input vector that combines syntactic and semantic information.
[0061] Reference Figure 3 The specific steps are as follows:
[0062] Step 301: Derive the syntactic weight vector according to the following principles.
[0063]
[0064] Where d i `seq_len` represents the syntactic distance between the i-th word and the aspect word, while `seq_len` represents the feature dimension size of the sentence. These are the calculated syntactic weights;
[0065] Step 302: Derive the part-of-speech weight vector according to the following principles.
[0066]
[0067] in This represents the part-of-speech weight of the i-th word. After obtaining the syntactic weight and part-of-speech weight, the syntactic and part-of-speech weights are normalized.
[0068] Step 303: Normalize the syntactic and part-of-speech weights to balance the differences in weight scales.
[0069]
[0070] Step 304: The normalized weights are combined by multiplication to obtain the final weight matrix.
[0071]
[0072] Where A m Based on w norm Mask matrix generated by weight vector
[0073] Step 305: Use a weighted edge enhancement strategy to handle special dependency relationships.
[0074] Define a weight matrix weights∈R n×n weights i,j This represents the weight between the i-th word and the j-th word;
[0075]
[0076]
[0077] In the formula, ε usually takes a minimum value, and if the word w i It is an adjective (ADJ), and w j If the noun in its dependency relation is NOUN, and the dependency relation label is "amod" modifying the relation, then it is defined as 0.8; if the word w i It is an adjective (ADV), and w j If the noun ADJ in its dependency relation is "amod", then it is defined as 0.7.
[0078] Step 306: By weighted summing of the mask vector and input features, the model focuses on the text portion directly related to the aspect words, and calculates the attention score and weights using ScaledDot-ProductAttention.
[0079]
[0080] Step 307: Then calculate the feature representation of the aspect words:
[0081] aspect = X·A m (12)
[0082] scores=linear(aspect)+Attention (13)
[0083] scores=softmax(scores·weights) (14)
[0084] Output = scores·V (15)
[0085] Step 308: Then, the outputs of each head are merged through a splicing operation and integrated into the final output through another linear layer, integrating information from multiple perspectives.
[0086] M=Concat(head1,head2,...,head h W O
[0087] W O It is another learnable parameter matrix, as shown in the following formula.
[0088] Using the syntactic and lexical enhanced graph attention network aspect-level sentiment analysis proposed in this paper, the performance of the proposed model was verified by comparing multiple models on four datasets and using accuracy and macro F1 score as evaluation metrics.
[0089] (1) Tests were conducted on four datasets: Twitter, Lap14, Rest14, and MAMS. The metrics were accuracy and macro F1 score.
[0090] (2) The models compared include G-ATT, AEN-BERT, BART-ABSA, BERT-SPC, ASGCN, CDT, SD-GCN, BiGCN, R-GAT, SSEGCN, SGAN and SLGAT.
[0091] The experimental results show that the SLGAT model outperforms most of the comparison models on the Twitter, Lap14, Rest14 and MAMS datasets. In particular, on the MAMS dataset, the accuracy and F1 score reached 0.8973 and 0.9411, respectively.
[0092] Ablation experiments were conducted on the Twitter, Lap14, Rest14, and MAMS datasets to systematically evaluate the three key components of the model: syntactic embedding, aspect word feature extraction, and weight enhancement. (Refer to...) Figure 4 :
[0093] (1) After removing syntactic embeddings, the accuracy and F1 score of all datasets generally decreased. For example, the accuracy of the Twitter dataset dropped from 77.31% to 76.73%, and the F1 score dropped from 76.23% to 75.79%.
[0094] (2) After removing aspect term features, the model performance declined, especially on the Restaurant dataset, where the F1 score dropped from 77.12% to 76.81%.
[0095] (3) After removing the weighted augmenting edges, the accuracy changed little on most datasets, but the F1 score dropped significantly on some datasets. For example, on the Laptop dataset, the F1 score dropped from 77.12% to 72.46%.
[0096] The impact of the number of graph attention network layers on model performance was evaluated on the Twitter, Lap14, Rest14, and MAMS datasets, respectively. (See reference...) Figure 5 :
[0097] (1) The four-layer GAT shows the best performance on the Lap14 dataset, with an accuracy of 78.05% and an F1 score of 80.34%. On the Twitter dataset, although the accuracy drops to 76.89%, the F1 score reaches 88.89%.
[0098] (2) The performance of the five-layer GAT generally declines. For example, in the MAMS dataset, the accuracy and F1 score drop to 76.64% and 76.09%, respectively.
[0099] The following analysis shows that the SLGAT model outperforms most comparable models on the Twitter, Lap14, Rest14, and MAMS datasets. On the MAMS dataset, it achieves an accuracy of 0.8973 and an F1 score of 0.9411. SLGAT enhances information richness and generalization ability while reducing noise impact by integrating syntactic and lexical information and using a weighted multi-head attention mechanism. On the Rest14 dataset, it slightly underperforms the R-GAT-BERT model, possibly because it ignores the subtle differences and complex interactions between syntactic and lexical information vectors. The four-layer GAT performs well on multiple datasets; a suitable number of layers is crucial for feature integration and extraction.
[0100] Example 2: Experimental Verification and Analysis
[0101] 2.1 Dataset and Evaluation Criteria
[0102] Four English benchmark datasets based on aspect-level sentiment analysis were used: the comment datasets from Lap14 and Res14 of SemEval-2014 Task 4, Twitter posts, and the MAMS dataset. The datasets cover three sentiment categories: positive, neutral, and negative. Each data point is labeled with one or more aspect items and a sentiment polarity label. The specific distribution is detailed in Table 1.
[0103] Table 1 Dataset Distribution
[0104]
[0105] On these datasets, two standard evaluation metrics, accuracy (Acc) and macro F1 score (MF1), were used to measure model performance, as shown in the formula.
[0106]
[0107] Where TP represents true positives, TN represents true negatives, FP represents false positives, and FN represents false negatives. The macro F1 score is the harmonic mean of precision and recall, calculated as follows:
[0108]
[0109] Where i represents the class index. The macro-average F1 score takes into account the F1 scores of all classes, providing a balanced measure of the model's performance in multi-class classification tasks, especially suitable for situations where the classes are unevenly distributed in the dataset.
[0110] 2.2 Experimental Setup
[0111] In the experimental setup of this study, Google's pre-trained model BERT-base-uncased was used to encode the text. The BERT dimension was set to 768. During training, the model used the RMSprop optimizer with a dropout ratio of 0.3 to prevent overfitting. Batch sizes of 16 and 48 samples were used in the training and validation phases, respectively. The model learning rate was 0.00002, and the L2 regularization was set to 0.0001 to prevent overfitting. Specific parameters are shown in Tables 2 and 3.
[0112] Table 2 Experimental Configuration
[0113]
[0114] Table 3 Experimental parameters
[0115]
[0116] 2.3 Comparative Experiment
[0117] ASGCN (Aspect-Specific Graph Convolution-alNetwork): This model integrates attention mechanisms with graph convolutional networks. It comprises two sub-models: ASGCN-DG and ASGCN-DT, which explore the different impacts of using directed and undirected graphs on text encoding.
[0118] CDT: The model uses graph convolutional networks to encode the grammatical structure, which enhances the semantic expression of the target and provides rich structured contextual information for sentiment classification tasks.
[0119] SD-GCN: Emphasizes the importance of aspect words in the analysis context, and extracts deep semantic features of text by combining attention mechanisms with GCN.
[0120] G-ATT combines memory networks and graph attention mechanisms to effectively integrate grammatical structure and local contextual information to identify key contextual words in text.
[0121] BiGCN: The model uses two parallel GCN models to simultaneously handle syntactic dependencies at the lexical level and word co-occurrence at the corpus level, achieving comprehensive capture of complex contextual information in the text.
[0122] R-GAT and R-GAT-BERT: These two models employ syntactic tree pruning techniques and apply a relation graph attention mechanism for efficient encoding. The R-GAT-BERT model further integrates BERT, significantly improving the model's performance on sentiment analysis tasks.
[0123] AEN-BERT: It combines standard attention mechanisms with multi-head attention, demonstrating powerful processing capabilities.
[0124] BART-ABSA: A unified generative model based on aspect sentiment analysis, specifically designed to comprehensively handle all subtasks of aspect-level sentiment analysis within an end-to-end framework.
[0125] BERT-SPC: The BERT-SPC model is a text pair classification framework that improves the accuracy of classification tasks by integrating aspect terms into the context and optimizing the construction of input sequences.
[0126] SSEGCN: Combining self-attention mechanism with aspect-aware attention mechanism, it significantly enhances the model's ability to deeply perceive text data.
[0127] SGAN: Combining a syntactic graph attention network, it incorporates dependency type knowledge into GAT, optimizes the attention weights of edges, and further improves the model's sensitivity to syntactic structure.
[0128] 2.4 Experimental Results and Analysis
[0129] The performance of the proposed model was validated by comparing these models on four datasets, using accuracy and macro F1 score as evaluation metrics. Specific results are shown in Table 4. These experiments demonstrate the efficiency and robustness of combining graph structure and attention mechanisms.
[0130] Table 2 reveals the impact of different word embedding techniques on model performance. While GloVe word embeddings excel at capturing word co-occurrence relationships, their statistical nature limits their ability to capture deep semantics. In contrast, pre-trained models based on BERT and its variants, such as AEN-BERT, BART-ABSA, and BERT-SPC, are trained on large-scale datasets, more effectively handle complex semantic information, and dynamically adjust word meanings based on context, significantly outperforming the G-ATT model using GloVe.
[0131] The ASGCN model significantly improves performance by using aspect word representations as attention weights. The CDT method effectively captures the semantic relationships between aspect words and opinion words by propagating context and dependency information, thus improving performance. While SD-GCN utilizes location information to obtain relevant context, its tree structure contains a large amount of useless information, resulting in significant interference. BiGCN successfully captures long-term dependencies between aspect words and opinion expressions by constructing a dependency-based graph and applying a bidirectional graph convolutional network. R-GAT, despite reconstructing the syntactic dependency tree, lacks external knowledge support. SSEGCN, combined with a sparse self-attention mechanism, effectively filters irrelevant words, improving performance. SGAN utilizes an attention mechanism to focus on the most relevant words and aspects when predicting sentiment polarity.
[0132] The SLGAT model outperforms most comparable models on the Twitter, Lap14, Rest14, and MAMS datasets, particularly on the MAMS dataset, where it achieves an accuracy of 0.8973 and an F1 score of 0.9411. This demonstrates the model's advantage in handling sentiment analysis tasks with large datasets. SLGAT effectively captures the relationship between aspect words and context by integrating syntactic and lexical information, as well as a weighted multi-head attention mechanism, enhancing information richness and generalization ability, and reducing the impact of noise through information filtering.
[0133] However, the SLGAT model performs slightly worse than the R-GAT-BERT model on the Rest14 dataset, possibly because SLGAT simply concatenates syntactic and lexical information vectors, ignoring the subtle differences and complex interactions between the two.
[0134] Table 4. Comparison results of different models (Note: ours represents the SLGAT model in this paper, the best result is in bold, "—" indicates that the original data is missing)
[0135] 2.5 Ablation Experiment
[0136] 2.5.1 Impact of Model Components
[0137] Ablation experiments were conducted on Twitter, lap14, Res14, and MAMS to systematically evaluate the three key components of the model: syntactic embedding, aspect word feature extraction, and weighted edge enhancement. Figure 4 The results of the Twitter / Lap14 / Rest14MAMS ablation experiment are shown in the figure.
[0138] Syntactic embeddings, which concatenate BERT embeddings and syntactic distance, generally lead to a decline in accuracy (ACC) and F1 scores across various datasets after this process is removed. For example, on the Twitter dataset, accuracy dropped from 77.31% to 76.73%, and the F1 score decreased from 76.23% to 75.79%. This change demonstrates that syntactic embeddings play a crucial role in enhancing the model's ability to understand sentence structure, thereby improving the accuracy and stability of sentiment classification.
[0139] Furthermore, removing the aspect-related feature extraction module resulted in a decline in model performance across different datasets, particularly on the Restaurant dataset, where the F1 score dropped from 77.12% to 76.81%. This result underscores the crucial role of aspect-related word feature extraction in accurately capturing and parsing target sentiment expression.
[0140] Finally, when the configuration of weighted augmenting edges was removed, although the accuracy changed little on most datasets, the F1 score dropped significantly on some datasets (e.g., the F1 score on the Laptop dataset dropped from 77.12% to 72.46%), indicating that weighted augmenting edges are crucial for the model to analyze inter-word interactions and their impact on overall sentiment.
[0141] 2.6 GAT Layer Analysis
[0142] 2.6.1 Impact of GAT Layer Number
[0143] To investigate the impact of the number of layers in a graph attention network (GCN) on model performance, the number of layers in the GCN was set to 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19 ... Figure 5 The experiment comparing the number of layers in Twitter GAT, Lap14GAT, Rest14 GAT, and MAMS GAT is shown.
[0144] The results show that GAT exhibits the best performance with a four-layer structure on the Lap14 dataset, achieving an accuracy of 78.05% and an F1 score of 80.34%. This indicates that a four-layer GAT effectively balances feature extraction and information integration, optimizing overall performance. Furthermore, when processing the Twitter dataset, although the accuracy slightly decreases to 76.89%, the F1 score remains high at 88.89%, demonstrating its excellent sentiment classification ability when analyzing complex social media text. However, when the number of layers increases to five, the performance of all datasets generally declines; for example, the accuracy and F1 score on the MAMS dataset drop to 76.64% and 76.09%, respectively. This performance degradation may stem from excessive feature fusion and gradient issues caused by too many layers, affecting model stability. Overall, the four-layer GAT performs well on multiple datasets, and a moderate number of layers is crucial for feature integration and extraction.
[0145] 2.6 Case Analysis
[0146] To evaluate the performance of the proposed model in practical applications, individual cases were selected from four datasets and compared with the classic models ASGCN, CDT, SSEGCN, and SLGAT. Detailed experimental results are shown in Table 5.
[0147] Sentence 1 contains the aspect word "service," which is close to the opinion word "dreadful," and their syntactic dependency is also quite obvious. Therefore, all models made correct predictions. Sentence 2 contains multiple aspect words, "fact" and "i5," with different aspect sentiment polarities. The first group of models mainly relied on context and attention mechanisms, failing to capture the sentiment features of specific aspects well, leading to incorrect judgments. Although the third group of models introduced some syntactic information, it still failed to completely solve this problem. The SLGAT model, by combining syntactic distance, syntactic weight, and lexical weight, can better guide the model to focus on opinion words related to aspect words and enrich the semantic features of the text, thus making correct predictions. In sentence 3, "Tequila" and "notbe disappointed" are far apart, and the sentiment expression is not obvious. The first three groups of models failed to capture the sentiment association correctly, resulting in prediction failures. The SLGAT model, through a multi-head attention mechanism, integrates and strengthens the feature representations and weighted edge embedding information related to aspect words, meticulously extracts the contextual information related to aspect words, and focuses on special syntactic dependencies, thus enabling correct predictions. In sentence 4, "keyboard" lacks a clear corresponding opinion word, and the first three models also failed to identify its sentiment tendency, leading to prediction failure. The SLGAT model, through its syntactically and lexically enhanced graph attention network, effectively extracts relevant feature representations of aspect words, enabling it to learn implicit sentiment from semantics to make correct judgments. Experimental results show that the SLGAT model significantly outperforms existing methods in aspect-level sentiment classification tasks on four benchmark datasets, further demonstrating its effectiveness and robustness.
[0148] Table 5 Case Prediction Results
[0149]
[0150] (Note: P, O, and N represent positive, neutral, and negative respectively; the terms have been bolded.)
[0151] Experiments demonstrate that the Syntactic-Lesson Enhanced Graph Attention Network (SLGAT) model of this invention is suitable for aspect-level sentiment analysis. This model fully utilizes syntactic and lexical information to improve the accuracy of sentiment analysis. The research designed aspect word feature extraction and weighted edge enhancement strategies, generating aspect word feature representations through syntactic and part-of-speech weights. In the graph attention network, a weighted multi-head attention mechanism is introduced to integrate aspect word features and weighted edge enhancement matrices, thereby more accurately capturing aspect-related information and significantly improving performance. Experimental results on four public datasets validate the model's effectiveness, highlighting the importance of syntactic and lexical information.
Claims
1. A sentiment analysis method based on a graph attention network model with syntactic and lexical enhancements, characterized in that, include: A. Design a graph attention network model that integrates syntactic distance, syntactic weight, and lexical weight into a dynamic weighted graph attention network, extract relevant feature representations of aspect words, and use a weighted edge enhancement strategy to handle special dependency relationships; B. By using a weighted multi-head attention mechanism, we integrate feature representations related to aspect words and weighted edge embedding information, and pay attention to special syntactic dependencies. The syntactic and lexical enhanced graph attention network is as follows: The shortest dependency syntactic path length and syntactic distance vector between each word and aspect word in the text are transformed through a linear layer, processed using a self-attention mechanism, and finally concatenated with the BERT embedding on the feature dimension. Simultaneously considering two dimensions, syntactic distance and part-of-speech category, syntactic weights and part-of-speech weights are generated. After normalization, the final weight matrix is generated by multiplication and combination, which explicitly expresses the syntactic and semantic relationships between words. Multi-dimensional feature information is extracted through a multi-head attention mechanism, and weighted summation is performed by combining mask vector and input features to focus on the text part related to aspect words; The weighted multi-head attention representation is input into a linear layer, and the linear output is transformed into a probability distribution through a softmax function, and the sentiment polarity is classified and output. The extraction of relevant feature representations of aspect words includes the following steps: B1. The syntactic weight vector is derived based on the following principles to express the relative importance of each word: （6）； B2, where the formula is Indicates the first The syntactic distance between individual words and aspect words, and This represents the size of the feature dimension of the sentence, while These are the calculated syntactic weights; B3. Calculate part-of-speech weights: （7）； B4, among which Indicates the first After determining the part-of-speech weights of each word and obtaining the syntactic and part-of-speech weights, normalization is performed on the syntactic and part-of-speech weights. （8）； （9）； B5. The normalized weights are combined by multiplication to obtain the final weight matrix. Then, a threshold is set to determine which weights are sufficient to indicate a high correlation between words and aspects. （10）； （11）； B6, among which Based on The mask matrix generated by the weight vector; The method of using weighted edge enhancement to handle special dependency relationships includes the following steps: Define a weight matrix , Indicates the first The word and the first Weights between words; （12）； （13）； In the formula It usually takes a very small value, and if the word It is an adjective (ADJ), and If the noun in its dependency relation is NOUN, and the dependency relation label is "amod" modifying the relation, then it is defined as 0.8; if the word It is an adjective (ADV), and If the noun ADJ in its dependency relation is "amod", then it is defined as 0.
7.
2. The sentiment analysis method based on a graph attention network model with syntactic and lexical enhancements as described in claim 1, characterized in that, Syntactic embedding is performed before graph attention network modeling, including the following steps: A1. Use the BERT pre-trained model to obtain the input sequence containing contextual semantic information: （1）； A2. Calculate the shortest dependency syntactic path length and syntactic distance vector between each word in the text and all words within the aspect phrase: （2）； A3, where the shortest dependency syntax path length is used To represent, and take the maximum value of these lengths, It refers to any word in the sentence other than the categorical noun. "This is a type of phrase"; A4. Use a self-attention mechanism to process the transformed syntactic distance vector, concatenating the BERT embeddings and self-attention outputs along the feature dimension: （3）； （4）； （5）； in Represents a linear layer. It is the dimension of the key. It is self-attention output. The "" symbol indicates a merge operation, which concatenates the text to obtain the final text representation. , This represents the final output, resulting in an input vector that combines syntactic and semantic information.
3. The sentiment analysis method based on a graph attention network model with syntactic and lexical enhancements as described in claim 1, characterized in that, The method integrates aspect-related feature representations and weighted edge embedding information through a weighted multi-head attention mechanism, including the following steps: D1. Input Data The query matrix is obtained through linear transformation. Key matrix Sum matrix , , and It is a learnable parameter matrix. This represents the index of the attention head, and the formula is as follows: （14）； （15）； （16）； D2. For each head, calculate the attention score and weights using Scaled Dot-Product Attention: （17）； D3, in the formula Let be the dimension of the key matrix; the scaling operation aims to prevent the inner product from becoming too large and affecting the gradient of the softmax function; the computation involves word feature representations: （18）； （19）； （20）； （21）； D4. Finally, the outputs of each head are merged through a concatenation operation and integrated into the final output through another linear layer, thus integrating information from multiple perspectives. It is another learnable parameter matrix, as shown in the following formula. （22）。