Aspect-level sentiment analysis method and system based on syntax graph and semantic sequence modeling

Abstract

The application provides an aspect-level sentiment analysis method and system based on syntax graph and semantic sequence modeling, and belongs to the field of natural language processing; the method comprises the following steps: performing semantic coding on an input sentence, generating a fused semantic representation after fusing part-of-speech embedding and relative distance embedding; the fused semantic representation is fused through multiple double-view alternately fused blocks stacked alternately, graph convolution operation is sequentially performed on a graph view in each block to aggregate syntax neighbor information and obtain graph enhanced representation, self-attention operation is performed on a sequence view to capture global semantic dependence and obtain sequence enhanced representation, syntax structure information is fused into semantic representation through one-way cross-view attention, and over-smoothing is relieved through a jump knowledge connection; finally, aspect-specific representation is outputted and sentiment polarity is predicted; through the double-view alternately fused mechanism, the syntax structure and the semantic sequence are cooperatively modeled, the over-smoothing problem of a deep network is effectively relieved, and the accuracy and robustness of fine-grained sentiment classification are improved.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and in particular to an aspect-level sentiment analysis method and system based on syntactic graphs and semantic sequence modeling. Background Technology

[0002] Aspect-Based Sentiment Analysis (ABSA) is a fine-grained text sentiment mining technique in the field of natural language processing. Its task is to identify the sentiment tendency of terms related to specific entity attributes or aspects in text. Compared with traditional discourse-level and sentence-level sentiment analysis, aspect-level sentiment analysis can accurately distinguish the sentiment polarity of multiple different aspect words in the same sentence.

[0003] For example, in the comment "The camera quality is excellent but the battery drains quickly," the sentiment towards "camera quality" is positive, while the sentiment towards "battery" is negative. This fine-grained analytical capability makes aspect-level sentiment analysis valuable in practical applications such as product reputation monitoring, user needs mining, and public opinion analysis.

[0004] In the process of performing aspect-level sentiment analysis tasks by computers, syntactic analysis is a crucial supporting technology for achieving accurate modeling. Syntactic analysis provides structured clues for matching aspect words with opinion words and accurately attributing sentiment polarity by analyzing the grammatical structure and dependency relationships between words in a sentence. Specifically, dependency parsing constructs a directed graph structure of the sentence using binary relationships of "headword-dependency word," which can intuitively reflect the grammatical relationships such as modification and subordination between words. For example, it can effectively match the aspect word "camera" scattered throughout the sentence with the opinion word "excellent" through dependency relationship chains, avoiding misattribution of sentiment attributes.

[0005] However, existing computer-executed aspect-level sentiment analysis methods still face the following technical challenges when using syntactic information for deep learning modeling:

[0006] First, the oversmoothing problem in deep graph network models significantly limits the computer's ability to represent fine-grained sentiment features. Most aspect-level sentiment analysis models based on graph convolutional networks, when stacking multiple layers to capture higher-order syntactic dependencies, experience homogenization of node features as the network depth increases, leading to a gradual loss of discriminative power in the representation vectors corresponding to different aspect words. The essence of this phenomenon lies in the fact that graph convolution operations can be viewed as a form of local smoothing filtering during computer execution. Repeated iterations cause node features to be "submerged" by neighboring information, making it difficult to retain their unique semantic attributes. In fine-grained sentiment analysis scenarios, oversmoothing directly prevents the computer from accurately distinguishing sentiment cues corresponding to multiple aspect words with different sentiment polarities within the same sentence, resulting in saturation or even a decline in classification performance.

[0007] Second, the modeling of graph views and sequence views is relatively independent during computer execution, lacking a deep collaborative interaction mechanism. Existing methods typically treat syntactic graph views and semantic sequence views as two isolated processing objects: one type of model only utilizes graph neural networks for information propagation on the syntactic graph, while the other type relies solely on Transformers to capture context on the sequence. Even when some works attempt to combine the two, they mostly employ simple sequential stacking or late-stage fusion strategies, failing to achieve deep coupling and collaborative learning between the two views. This loosely coupled approach causes computers to ignore the intrinsic connection between syntactic structure and semantic sequence when performing sentiment analysis: syntactic dependencies are often implicit in word order, while semantic understanding requires syntactic structure as a framework.

[0008] Third, single-architecture models have inherent limitations when executed by computers, making it difficult to simultaneously meet the dual modeling requirements of syntactic structure and sequence semantics. Some models rely solely on graph neural networks to capture syntactic structure information. While they can effectively utilize dependency constraints, they are limited by the scope of local message passing and struggle to capture global semantic relationships that span syntactic distances. Other models rely solely on Transformers to capture long-distance dependencies. Although they can model the global context, they lack the ability to explicitly model syntactic structure, making them prone to mismatches between aspect words and opinion words in complex sentences. This predicament is particularly pronounced when processing comment texts containing multiple aspect words, long-distance dependencies, and complex logical relationships, thus limiting the accuracy of the sentiment classification results output by the computer.

[0009] Therefore, there is an urgent need to propose a new technical solution to address the technical problems in existing computer-executed aspect-level sentiment analysis methods, such as the loss of distinguishability of word node representations in different aspects due to the oversmoothing problem of deep graph convolutional networks, the lack of deep collaborative interaction between independent modeling of syntactic graph views and semantic sequence views, and the inability of a single architecture to simultaneously meet the dual modeling requirements of local syntactic dependencies and global semantic associations, resulting in insufficient accuracy and robustness of sentiment classification results. Summary of the Invention

[0010] To address the aforementioned technical problems, this invention provides an aspect-level sentiment analysis method and system based on syntactic graph and semantic sequence modeling. The aim is to improve the accuracy of fine-grained sentiment classification by using a dual-view alternating fusion mechanism. This mechanism enables the computer to perform graph convolution operations on the syntactic graph view to strengthen the syntactic dependency paths between aspect words and opinion words, and to perform sequence transformation operations on the semantic sequence view to capture long-distance semantic associations. Furthermore, it achieves deep coupling and oversmoothing suppression of the two views through a unidirectional cross-view attention mechanism and skip knowledge connections.

[0011] To achieve the above objectives, this application proposes an aspect-level sentiment analysis method based on syntactic graphs and semantic sequence modeling, executed by a computer, for predicting the sentiment polarity of a given target aspect word in an input sentence, including the following steps:

[0012] Text encoding and position encoding steps: Perform basic semantic encoding on the input sentence to obtain the initial semantic representation of the input sentence; Based on the dependency parsing tree obtained from the dependency parsing of the input sentence and the relative positions of the target aspect words, generate structural position encoding, which includes part-of-speech embedding and relative distance embedding; Fuse the initial semantic representation with the structural position encoding to generate a fused semantic representation;

[0013] The alternating dual-view fusion step involves using the fusion semantic representation as initial input and processing it sequentially through multiple alternately stacked dual-view fusion blocks. Each dual-view fusion block performs operations in the following order:

[0014] First, on the syntactic graph view, based on the adjacency matrix constructed from the dependency syntax tree and storing dependency relation types, a graph convolution operation based on dense message passing is performed on the input representation to aggregate the neighbor node information that has a dependency relation with the current node, thereby obtaining a graph-enhanced representation;

[0015] Secondly, on the semantic sequence view, a self-attention-based sequence transformation operation is performed on the input representation to capture global semantic dependencies and obtain a sequence-enhanced representation;

[0016] Finally, a one-way cross-view attention operation is performed, using the sequence augmented representation as the query and the graph augmented representation as the key and value. The syntactic structure information in the graph augmented representation is integrated into the sequence augmented representation through the attention mechanism to generate a cross-view fusion representation, without performing the reverse cross-view attention operation that uses the graph augmented representation as the query.

[0017] The cross-view fusion representation is residually connected to the input representation of the dual-view alternating fusion block. This residual connection constitutes a jump knowledge connection from the block input to the block output to alleviate oversmoothing, thereby obtaining the output representation of the dual-view alternating fusion block.

[0018] Sentiment classification output steps: Based on the final output representation of the alternating stacked dual-view alternating fusion blocks, extract the aspect-specific representation of the target aspect word, and predict the sentiment polarity of the target aspect word accordingly.

[0019] As a further solution, the text encoding and position encoding steps specifically include:

[0020] The input sentence is context-encoded using the pre-trained language model BERT to obtain the initial semantic representation and a classification tag vector CLS located at the beginning of the sentence to represent the global semantics at the sentence level.

[0021] As a further solution, before generating the fused semantic representation, the following is also included:

[0022] The initial semantic representation is input into a feature adaptation layer for processing. The feature adaptation layer consists of two linear transformation layers and a GELU activation function located between them.

[0023] As a further solution, the method for generating the fused semantic representation in the text encoding and position encoding steps is as follows:

[0024] The initial semantic representation, the part-of-speech embedding, and the relative distance embedding are concatenated, and then dimensional mapping is performed through a linear projection layer to obtain the fused semantic representation.

[0025] The part-of-speech embedding is obtained by mapping part-of-speech tags obtained by performing dependency parsing on the input sentence; the relative distance embedding is obtained by calculating the relative distance between each word in the input sentence and the target aspect word, and then mapping it after discretization.

[0026] As a further solution, the method for generating the fused semantic representation in the text encoding and position encoding steps is as follows:

[0027] The semantic representation processed by the feature adaptation layer, the part-of-speech embedding, and the relative distance embedding are concatenated, and then dimensional mapping is performed through a linear projection layer to obtain the fused semantic representation.

[0028] The part-of-speech embedding is obtained by mapping part-of-speech tags obtained by performing dependency parsing on the input sentence; the relative distance embedding is obtained by calculating the relative distance between each word in the input sentence and the target aspect word, and then mapping it after discretization.

[0029] As a further solution, the graph convolution operation based on dense message passing is specifically a text-oriented dense batch graph convolution operation, including:

[0030] Based on the adjacency matrix, edge feature embeddings are generated for the edges that represent dependency relationship types.

[0031] The input representation is expanded along the row direction, added to the edge feature embedding, and activated by ReLU to obtain the original message matrix;

[0032] Generate a mask based on the adjacency matrix and set the messages at non-edge positions in the original message matrix to zero;

[0033] For the message matrix after zeroing, for each node, aggregate the messages of all its neighbors along the neighbor dimension to obtain the aggregated representation of the node, and perform a nonlinear transformation through a node update function implemented by a multilayer perceptron to obtain the graph augmented representation;

[0034] Furthermore, the edge feature embedding is added to the node features extended along the row direction and the node features extended along the column direction, and then subjected to a nonlinear transformation through an edge update function implemented by a multilayer perceptron to obtain the updated edge feature representation.

[0035] As a further solution, in the unidirectional cross-view attention operation, the sequence augmentation representation and the graph augmentation representation are mapped to queries, keys and values ​​by multiplying with their respective learnable projection matrices, and attention weights are calculated.

[0036] As a further solution, the emotion classification output step specifically includes:

[0037] Average pooling is performed on the final output representation of each word contained in the target aspect word to obtain the aspect-specific representation of the target aspect word;

[0038] The aspect-specific representation is input into a classifier to predict the sentiment polarity probability distribution of the target aspect word.

[0039] As a further solution, the emotion classification output step specifically includes:

[0040] Average pooling is performed on the final output representation of each word contained in the target aspect word to obtain the aspect-specific representation of the target aspect word;

[0041] The aspect-specific representation is concatenated with the classification label vector CLS corresponding to the input sentence to obtain the classification input vector.

[0042] The classification input vector is fed into a fully connected layer and a Softmax function to predict the sentiment polarity probability distribution of the target aspect words.

[0043] On the other hand, the present invention also provides an aspect-level sentiment analysis system based on syntactic graph and semantic sequence co-modeling, comprising:

[0044] Memory, used to store the computer programs required to build aspect-level sentiment analysis models;

[0045] A processor, configured to execute computer programs stored in the memory to enable the operation of various modules of the aspect-level sentiment analysis model, including:

[0046] The text encoding and position encoding module includes: a text encoder for semantic encoding of the input sentence; and a position encoding fusion layer for fusing structural position encodings generated based on a dependency syntax tree to output a fused semantic representation.

[0047] Multiple alternately stacked dual-view alternating fusion modules, each of the dual-view alternating fusion modules comprising:

[0048] The graph convolutional layer based on dense message passing is used to aggregate syntactic neighbor information on the graph view. It includes an edge feature embedding unit, a message construction unit, a mask filtering unit, a node message aggregation unit, a node update unit, and an edge update unit.

[0049] A self-attention-based sequence transformation layer is used to capture global semantic dependencies on the sequence view;

[0050] A cross-view attention layer is used to perform unidirectional attention calculations using the output of the sequence transformation layer as a query and the output of the graph convolution layer as a key and value to achieve information fusion between views.

[0051] The residual connection unit is used to add the output of the cross-view attention layer to the input representation of the dual-view alternating fusion module by residual addition to form a jump knowledge connection to alleviate oversmoothing.

[0052] The pooling and sentiment classification module includes: an aspect-specific representation pooling layer for extracting aspect-specific representations of target aspect words from the output of the last dual-view alternating fusion module; and a sentiment classifier for predicting sentiment polarity based on the aspect-specific representations.

[0053] Compared with related technologies, the aspect-level sentiment analysis method and system based on syntactic graphs and semantic sequence modeling provided by this invention has the following advantages:

[0054] 1. This invention models text simultaneously as a syntactic graph view and a semantic sequence view. Through multiple alternately stacked dual-view fusion blocks, graph convolution and sequence transformation operations are performed sequentially within each block, allowing syntactic dependencies and global semantic information to mutually enhance each other in a layer-by-layer alternating process. Unlike existing methods that isolate the two views or simply fuse them later, this invention uses a unidirectional cross-view attention mechanism to precisely integrate the syntactic structure information from the graph-enhanced representation into the sequence-enhanced representation, achieving deep coupling of the two views at the feature level and effectively compensating for the information loss caused by modeling a single view.

[0055] 2. This invention forces node representations to switch and update between different views by alternating between graph convolution and sequence transformation, avoiding the homogenization of node features caused by repeated local smoothing operations in a single view. Simultaneously, residual connections within each dual-view alternating fusion block constitute skip knowledge connections, preserving the discriminative information of the underlying features. Ablation experiments show that the complete model of this invention achieves an accuracy of 85.02% and an F1 score of 79.66% on the Restaurant dataset, representing improvements of 0.98% and 2.32% respectively compared to the variant that removes cross-view attention, validating the effectiveness of the above design in suppressing over-smoothing and maintaining feature discriminativeness.

[0056] 3. This invention outperforms mainstream benchmark models on three publicly available datasets: Restaurant, Laptop, and Twitter. Without BERT, this invention achieves a 0.75% improvement in accuracy and a 1.58% improvement in F1 score compared to the suboptimal DualGCN model on the Restaurant dataset. With BERT, the accuracy and F1 score reach 87.29% and 81.93%, respectively. Particularly on samples containing multi-faceted words and complex sentence structures, this invention, through the synergistic effect of syntactic dependency path reinforcement and global semantic capture, can more accurately distinguish the sentiment polarity of words with different facets, demonstrating stronger robustness.

[0057] 4. Compared with the pure sequence transformation structure, the total number of parameters of the present invention increases by only 0.15M, the number of trainable parameters increases by 148.6K, and the average inference time increases by only 0.16ms. It achieves dual-view collaborative modeling capability with minimal computational overhead, realizing a good balance between accuracy and efficiency, and has good potential for practical application. Attached Figure Description

[0058] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0059] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 A schematic diagram illustrating the steps of an aspect-level sentiment analysis method based on syntactic graphs and semantic sequence modeling provided by this invention;

[0061] Figure 2 The model processing flowchart provided by this invention;

[0062] Figure 3 Examples of oversmoothing and fusion of different models are shown; (a) is an example of oversmoothing of GCN; (b) is an example of oversmoothing of Transformer; and (c) is an example of alternating fusion of GCN and Transformer.

[0063] Figure 4 The accuracy of the model in the test embodiment of this invention varies with the number of alternating fusion block layers of dual views;

[0064] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0066] Example 1

[0067] Please see Figure 1 This embodiment provides an aspect-level sentiment analysis method based on syntactic graphs and semantic sequence modeling, executed by a computer, for predicting the sentiment polarity of a given target aspect word in an input sentence, including the following steps:

[0068] Text encoding and position encoding steps: Perform basic semantic encoding on the input sentence to obtain the initial semantic representation of the input sentence; Based on the dependency parsing tree obtained from the dependency parsing of the input sentence and the relative positions of the target aspect words, generate structural position encoding, which includes part-of-speech embedding and relative distance embedding; Fuse the initial semantic representation with the structural position encoding to generate a fused semantic representation;

[0069] The alternating dual-view fusion step involves using the fusion semantic representation as initial input and processing it sequentially through multiple alternately stacked dual-view fusion blocks. Each dual-view fusion block performs operations in the following order:

[0070] First, on the syntactic graph view, based on the adjacency matrix constructed from the dependency syntax tree and storing dependency relation types, a graph convolution operation based on dense message passing is performed on the input representation to aggregate the neighbor node information that has a dependency relation with the current node, thereby obtaining a graph-enhanced representation;

[0071] Secondly, on the semantic sequence view, a self-attention-based sequence transformation operation is performed on the input representation to capture global semantic dependencies and obtain a sequence-enhanced representation;

[0072] Finally, a one-way cross-view attention operation is performed, using the sequence augmented representation as the query and the graph augmented representation as the key and value. The syntactic structure information in the graph augmented representation is integrated into the sequence augmented representation through the attention mechanism to generate a cross-view fusion representation, without performing the reverse cross-view attention operation that uses the graph augmented representation as the query.

[0073] The cross-view fusion representation is residually connected to the input representation of the dual-view alternating fusion block. This residual connection constitutes a jump knowledge connection from the block input to the block output to alleviate oversmoothing, thereby obtaining the output representation of the dual-view alternating fusion block.

[0074] Sentiment classification output steps: Based on the final output representation of the alternating stacked dual-view alternating fusion blocks, extract the aspect-specific representation of the target aspect word, and predict the sentiment polarity of the target aspect word accordingly.

[0075] It should be noted that this embodiment aims to accurately predict the sentiment polarity of a given target aspect word in an input sentence. First, the input sentence is subjected to basic semantic encoding through text encoding and position encoding steps to obtain an initial semantic representation. At the same time, a dependency parsing tree is constructed based on dependency parsing, and a structural position encoding containing part-of-speech embedding and relative distance embedding is generated by combining the relative position of the target aspect word. The two are then fused to form a fused semantic representation, thereby explicitly injecting syntactic structure information during the encoding stage.

[0076] The core step then proceeds to alternating dual-view fusion, where the fusion semantic representation is sequentially input into multiple alternately stacked dual-view fusion blocks for processing. Within each fusion block, three types of operations are performed in a fixed order: image first, sequence second.

[0077] First, in the syntactic graph view, based on the adjacency matrix constructed from the dependency syntax tree and storing dependency relation types, a graph convolution operation based on dense message passing is performed on the input representation to aggregate the dependency neighbor information of the current node to obtain a graph-enhanced representation. Second, in the semantic sequence view, a sequence transformation operation based on self-attention is performed on the same input representation to capture global semantic dependencies to obtain a sequence-enhanced representation. Finally, a one-way cross-view attention operation is performed, using the sequence-enhanced representation as the query and the graph-enhanced representation as the key and value, integrating syntactic structure information into the semantic representation to form a cross-view fusion representation, without performing a reverse attention operation. The input and output of the fusion block form a skip knowledge connection through residual connections to alleviate the oversmoothing problem of deep networks.

[0078] After deep processing through multiple fusion blocks, the process proceeds to the sentiment classification output step. Based on the final output representation, aspect-specific pooling is performed on the word vectors contained in the target aspect word to predict the sentiment polarity of that aspect word and output the classification result. This scheme achieves deep fusion of local syntactic constraints and global semantic information through alternating collaborative modeling of syntactic graph views and semantic sequence views, effectively improving the accuracy of fine-grained sentiment classification.

[0079] Furthermore, the text encoding and position encoding steps specifically include:

[0080] The input sentence is context-encoded using the pre-trained language model BERT to obtain the initial semantic representation and a classification tag vector CLS located at the beginning of the sentence to represent the global semantics at the sentence level.

[0081] Specifically, in this embodiment, when encoding the input sentence, BERT utilizes its multi-layer Transformer structure to model the sentence context, generating an initial semantic representation for each word that incorporates contextual information. This allows word vectors to dynamically reflect their meaning in a specific context, rather than being static, fixed word vectors. Furthermore, BERT automatically adds a special classification label vector (CLS) at the beginning of the input sequence. After undergoing multi-layer self-attention computation, this vector aggregates the global semantic information of the entire sentence, serving as a compact sentence-level representation.

[0082] Therefore, this embodiment has significant advantages compared to general word vector methods: BERT is pre-trained on large-scale unsupervised corpora and has accumulated rich linguistic knowledge and experience in sentiment semantic modeling, enabling it to automatically identify sentiment keywords and their modification range in text. Simultaneously, the [CLS] vector output by BERT provides sentence-level global semantic features for subsequent sentiment classification output steps. When concatenated with aspect-specific representations of target aspect words, it further enhances the classifier's ability to perceive contextual sentiment, allowing the model to maintain stable recognition accuracy when dealing with complex texts with multiple aspects and sentiment polarities.

[0083] Furthermore, before generating the fused semantic representation, the process also includes:

[0084] The initial semantic representation is input into a feature adaptation layer for processing. The feature adaptation layer consists of two linear transformation layers and a GELU activation function located between them.

[0085] Specifically, while the initial semantic representation output by BERT contains rich contextual information, its feature distribution is formed based on the pre-training task and may not fully fit the feature space of the downstream task of aspect-level sentiment analysis. By performing a non-linear mapping on the BERT output through a feature adaptation layer, the representation can be adjusted to a feature distribution that better suits the needs of subsequent dual-view alternating fusion block processing while keeping the feature dimensions unchanged, thereby improving information transmission efficiency.

[0086] In choosing the activation function, this embodiment uses GELU instead of the traditional ReLU, which has clear technical considerations. GELU has the characteristic of being smooth and continuous across the entire domain, and its gradient does not completely vanish in the negative value region, which can effectively ensure the stable propagation of gradients in deep networks. At the same time, GELU outputs a non-zero small value for negative inputs, which can retain more potential effective semantic information, especially suitable for scenarios in sentiment analysis where negative words, transition words, etc., need to convey semantics through negative values.

[0087] In addition, the feedforward networks of mainstream pre-trained models such as BERT all use GELU as the activation function. Using this function in the feature adaptation layer can maintain consistency with the feature distribution output by BERT, reduce the risk of representation drift, and make the adapted semantic representation more stable in the subsequent structural position encoding fusion and dual-view alternating fusion stages.

[0088] Furthermore, in the text encoding and position encoding steps, the method for generating the fused semantic representation is as follows:

[0089] The initial semantic representation, the part-of-speech embedding, and the relative distance embedding are concatenated, and then dimensional mapping is performed through a linear projection layer to obtain the fused semantic representation.

[0090] The part-of-speech embedding is obtained by mapping part-of-speech tags obtained by performing dependency parsing on the input sentence; the relative distance embedding is obtained by calculating the relative distance between each word in the input sentence and the target aspect word, and then mapping it after discretization.

[0091] Specifically, after obtaining the initial semantic representation output by BERT, the model does not directly use it as input for subsequent processing. Instead, it concatenates the initial semantic representation with two structural positional encodings—part-of-speech embedding and relative distance embedding—and then performs dimensional mapping through a linear projection layer to finally form a fused semantic representation.

[0092] This embodiment enables the final representation vector to simultaneously carry three types of information: deep contextual semantics from BERT, grammatical role information from dependency parsing, and spatial position information of words relative to the target aspect words, thus achieving explicit fusion of semantic features and syntactic structural features at the vector level.

[0093] Part-of-speech (POS) embeddings rely on the results of dependency parsing: during POS parsing of the input sentence, the parser labels each word with its part-of-speech tag, such as noun, verb, or adjective. These tags contain rich grammatical role information, helping the model distinguish between aspect words (usually nouns) and opinion words (usually adjectives or verbs). By assigning a learnable embedding vector to each POS tag and performing a lookup table mapping, the model can automatically learn the semantic contribution of different POS tags to sentiment expression during training. Relative distance embeddings are generated by calculating the relative distance between each word in the input sentence and the target aspect word, i.e., the word's position offset in the sequence, and then discretizing this distance value and mapping it to the corresponding embedding vector.

[0094] Therefore, this embodiment enables the model to perceive the spatial proximity of each word to the target aspect word. The closer the word is to the context word of the target aspect word, the stronger its sentiment modification effect is usually, thus providing explicit positional prior knowledge for subsequent graph convolution and sequence transformation operations.

[0095] Furthermore, in the text encoding and position encoding steps, the method for generating the fused semantic representation is as follows:

[0096] The semantic representation processed by the feature adaptation layer, the part-of-speech embedding, and the relative distance embedding are concatenated, and then dimensional mapping is performed through a linear projection layer to obtain the fused semantic representation.

[0097] The part-of-speech embedding is obtained by mapping part-of-speech tags obtained by performing dependency parsing on the input sentence; the relative distance embedding is obtained by calculating the relative distance between each word in the input sentence and the target aspect word, and then mapping it after discretization.

[0098] Specifically, in this embodiment, the semantic representation processed by the feature adaptation layer is concatenated with two structural positional encodings: part-of-speech embedding and relative distance embedding. Then, a linear projection layer is used for dimensional mapping to finally obtain the fused semantic representation. Therefore, the feature adaptation layer has adjusted the BERT output to a feature distribution that is more suitable for downstream tasks. On this basis, syntactic structure information is injected so that the positional encoding can work better with the adapted semantic features.

[0099] The generation methods for part-of-speech embedding and relative distance embedding are the same as described above: part-of-speech embedding is obtained by mapping the part-of-speech tags obtained from dependency parsing of the input sentence, which can provide the model with grammatical role information for each word; relative distance embedding is obtained by calculating the relative distance between each word in the input sentence and the target aspect word and then mapping it after discretization, which can provide the model with the spatial position awareness of words relative to the target aspect word.

[0100] Therefore, this embodiment combines the adapted deep semantic features with grammatical role features and spatial distance features at the vector level. Before entering the dual-view alternating fusion step, the model already has multi-dimensional information that perceives grammatical structure, semantic context, and the relative positions of aspect words, providing richer input representations for subsequent graph convolution operations and sequence transformation operations.

[0101] Furthermore, the graph convolution operation based on dense message passing is specifically a text-oriented dense batch graph convolution operation, including:

[0102] Based on the adjacency matrix, edge feature embeddings are generated for the edges that represent dependency relationship types.

[0103] The input representation is expanded along the row direction, added to the edge feature embedding, and activated by ReLU to obtain the original message matrix;

[0104] Generate a mask based on the adjacency matrix and set the messages at non-edge positions in the original message matrix to zero;

[0105] For the message matrix after zeroing, for each node, aggregate the messages of all its neighbors along the neighbor dimension to obtain the aggregated representation of the node, and perform a nonlinear transformation through a node update function implemented by a multilayer perceptron to obtain the graph augmented representation;

[0106] Furthermore, the edge feature embedding is added to the node features extended along the row direction and the node features extended along the column direction, and then subjected to a nonlinear transformation through an edge update function implemented by a multilayer perceptron to obtain the updated edge feature representation.

[0107] Specifically, the core process of this embodiment includes:

[0108] First, based on the adjacency matrix constructed from the dependency syntax tree and storing dependency relation types, a learnable embedding layer generates edge feature embeddings for each edge, transforming discrete dependency relation types into dense vector representations. This enables the model to distinguish the differentiated impact of different grammatical relations (such as subject-verb, verb-object, adjective modification, etc.) on sentiment expression. Second, the input node representation is expanded along the row direction, added to the edge feature embedding, and activated using ReLU to construct the original message matrix corresponding to each edge. Subsequently, a mask is generated based on the adjacency matrix, setting all messages at non-edge positions to zero.

[0109] The masking filtering step is the key to the entire operation: it ensures that only words with real dependencies can pass messages, eliminating noise information from interfering with node representation updates, thereby guaranteeing the purity of the graph structure information.

[0110] In the message aggregation and update phase, this embodiment employs a separate node update and edge update dual-path mechanism. For the zeroed message matrix, for each node, the messages of all its neighboring nodes are aggregated along the neighbor dimension to obtain the aggregated representation of that node. This is then transformed nonlinearly through a node update function implemented by a multilayer perceptron to generate the final graph-enhanced representation. Simultaneously, the edge feature embedding is added to the node features extended along the row and column directions, and transformed nonlinearly through another edge update function implemented by a multilayer perceptron to obtain the updated edge feature representation for use in the next dual-view alternating fusion block. This design of updating nodes and edges separately allows edge features to adaptively adjust according to the representations of the nodes they connect to, thereby continuously optimizing the representation capability of syntactic dependency paths during multi-layer stacking.

[0111] Compared to traditional Graph Convolutional Networks (GCNs), the graph convolution operation in this embodiment has two significant advantages:

[0112] First, message passing is achieved through dense batch processing, avoiding the difficulty of parallelizing sparse graph operations in batch processing scenarios and significantly improving computational efficiency. Second, through mask filtering and multilayer perceptron update functions, both strict constraints on the graph structure are ensured, and the model is endowed with sufficient nonlinear expressive power, effectively alleviating the oversmoothing problem commonly found in multilayer graph convolution. This operation, in conjunction with subsequent sequence transformation operations and cross-view attention operations, achieves deep fusion of syntactic graph views and semantic sequence views.

[0113] Furthermore, in the unidirectional cross-view attention operation, the sequence augmentation representation and the graph augmentation representation are mapped to queries, keys, and values ​​by multiplying with their respective learnable projection matrices, and attention weights are calculated.

[0114] Specifically, sequence augmentation represents generating a query vector by multiplying it with the query projection matrix, which expresses the retrieval intent of "what information needs to be obtained from the syntactic structure at the current semantic position"; graph augmentation represents generating a key vector and a value vector by multiplying it with the key projection matrix and the value projection matrix, respectively. The key vector is used to match the query to calculate attention weights, while the value vector serves as the actual information content to be aggregated.

[0115] Since sequence augmentation representations originate from self-attention-based sequence transformation operations, they contain global contextual semantic information; while graph augmentation representations originate from graph convolution operations based on dense message passing performed on syntactic graph views, they contain structured information about local syntactic dependencies. These two representations reside in different feature spaces. If attention is calculated directly without independent linear projections, the mismatch in feature spaces will prevent attention weights from accurately reflecting the correlation between syntactic structure and semantic requirements. By introducing three sets of learnable projection matrices, the model can automatically learn during training how to align the two heterogeneous representations to a common space suitable for attention interaction, thereby calculating more meaningful attention weights.

[0116] After completing the projection mapping, the model calculates attention weights to determine the importance of each node's information in the graph augmented representation to its position in the sequence augmented representation. A higher weight indicates that the information of that syntactic node is more relevant to the current semantic position. Finally, the model performs weighted aggregation of the value vectors based on the attention weights, accurately integrating the syntactic structure information from the graph augmented representation into the sequence augmented representation, generating a semantic representation that incorporates structured grammatical constraints.

[0117] Therefore, the one-way design in this embodiment uses only sequence augmentation representation for queries and graph augmentation representation for keys and values, without performing reverse operations, ensuring the guiding role of syntactic information in semantic understanding, while avoiding redundant bidirectional computational overhead.

[0118] Furthermore, the emotion classification output step specifically includes:

[0119] Average pooling is performed on the final output representation of each word contained in the target aspect word to obtain the aspect-specific representation of the target aspect word;

[0120] The aspect-specific representation is input into a classifier to predict the sentiment polarity probability distribution of the target aspect word.

[0121] Specifically, after processing multiple alternating dual-view fusion blocks, the model obtains the final output representation of each word, which simultaneously contains syntactic structural information and global semantic information. For a given target aspect word, since the aspect word may be composed of multiple words (e.g., "camera quality" is composed of two words), this embodiment first performs average pooling on the final output representations corresponding to these constituent words, compressing these word vectors into a fixed-dimensional aspect-specific representation vector. This pooling method can equally fuse the semantic information of each constituent word within the aspect word, avoiding the problem of inconsistent representation dimensions caused by the different lengths of aspect words, while retaining the semantic features most critical for sentiment judgment.

[0122] Subsequently, in this embodiment, the aspect-specific representation obtained through pooling is directly input into the classifier to predict the sentiment polarity probability distribution of the target aspect word and output the sentiment classification result. This direct classification design is suitable for scenarios using non-BERT encoders such as BiLSTM. In this scenario, the aspect-specific representation has already incorporated contextual semantics and syntactic structure information, and accurate sentiment polarity judgment can be achieved without introducing additional sentence-level global features. The classifier outputs the sentiment polarity probability distribution, that is, the model's prediction confidence for positive, negative, and neutral sentiment categories. Finally, the category with the highest probability is output as the sentiment classification result for the target aspect word.

[0123] Furthermore, the emotion classification output step specifically includes:

[0124] Average pooling is performed on the final output representation of each word contained in the target aspect word to obtain the aspect-specific representation of the target aspect word;

[0125] The aspect-specific representation is concatenated with the classification label vector CLS corresponding to the input sentence to obtain the classification input vector.

[0126] The classification input vector is fed into a fully connected layer and a Softmax function to predict the sentiment polarity probability distribution of the target aspect words.

[0127] Specifically, after processing multiple alternating dual-view fusion blocks, the model first performs average pooling on the final output representations of each word contained in the target aspect word, generating a fixed-dimensional aspect-specific representation vector. This vector fuses the syntactic and semantic features of each constituent word within the aspect word. This embodiment adds a crucial concatenation operation: concatenating the aspect-specific representation with the classification label vector CLS corresponding to the input sentence to form a classification input vector. During the encoding phase, BERT generates a classification label vector CLS for each input sentence, located at the beginning of the sentence. After multiple layers of self-attention computation, this vector can aggregate the global semantic information of the entire sentence, reflecting the overall sentiment and semantic background of the sentence.

[0128] In fine-grained sentiment analysis tasks, the sentiment polarity of a target aspect word depends not only on its own semantic features and local context, but also on the semantic atmosphere and global logical relationships of the entire sentence. For example, in sentences containing contrastive relationships, relying solely on local information near the aspect word may not accurately determine its sentiment polarity. By concatenating aspect-specific representations with the classification label vector CLS, the classifier can simultaneously acquire the local semantic features of both aspect words and the global semantic information of the sentence, thus making sentiment judgments within a more complete semantic context. Finally, the concatenated classification input vector is passed through a fully connected layer and a Softmax function, outputting the probability distribution of the target aspect word across three sentiment categories: positive, negative, and neutral. The category with the highest probability is taken as the final sentiment classification result.

[0129] Example 2

[0130] This embodiment also provides an aspect-level sentiment analysis system based on syntactic graph and semantic sequence co-modeling, including:

[0131] Memory, used to store the computer programs required to build aspect-level sentiment analysis models;

[0132] A processor, configured to execute computer programs stored in the memory to enable the operation of various modules of the aspect-level sentiment analysis model, including:

[0133] The text encoding and position encoding module includes: a text encoder for semantic encoding of the input sentence; and a position encoding fusion layer for fusing structural position encodings generated based on a dependency syntax tree to output a fused semantic representation.

[0134] Multiple alternately stacked dual-view alternating fusion modules, each of the dual-view alternating fusion modules comprising:

[0135] The graph convolutional layer based on dense message passing is used to aggregate syntactic neighbor information on the graph view. It includes an edge feature embedding unit, a message construction unit, a mask filtering unit, a node message aggregation unit, a node update unit, and an edge update unit.

[0136] A self-attention-based sequence transformation layer is used to capture global semantic dependencies on the sequence view;

[0137] A cross-view attention layer is used to perform unidirectional attention calculations using the output of the sequence transformation layer as a query and the output of the graph convolution layer as a key and value to achieve information fusion between views.

[0138] The residual connection unit is used to add the output of the cross-view attention layer to the input representation of the dual-view alternating fusion module by residual addition to form a jump knowledge connection to alleviate oversmoothing.

[0139] The pooling and sentiment classification module includes: an aspect-specific representation pooling layer for extracting aspect-specific representations of target aspect words from the output of the last dual-view alternating fusion module; and a sentiment classifier for predicting sentiment polarity based on the aspect-specific representations.

[0140] The model processing flow in this embodiment (as follows) Figure 2 As shown, it is mainly divided into a text encoding and position encoding layer, a dual-view alternating fusion block stacking layer, and a pooling and classification output layer.

[0141] First, at the text encoding and position encoding layers, the model performs initial semantic encoding on the input sentence using BiLSTM or BERT, while fusing part-of-speech embedding and relative distance embedding as structural position encoding, injecting explicit syntactic structure information into the underlying layer of the model, laying the foundation for subsequent dual-view learning.

[0142] Secondly, the stacked layer of alternating dual-view fusion blocks is the core of the entire model. Within each alternating dual-view fusion block, the following operations are executed sequentially: Graph convolution operation: On the syntactic graph view, based on an adjacency matrix constructed from a dependency syntax tree and storing dependency relation types, a dense message-passing-based graph convolution operation is performed on the input representation to aggregate neighboring node information that is dependent on the current node, mining the dependency relationships between aspect terms and view terms; Sequence transformation operation: On the semantic sequence view, a self-attention-based sequence transformation operation is performed on the input representation to capture global semantic associations; Cross-view attention operation: Using the sequence-enhanced representation as a query and the graph-enhanced representation as a key and value, a unidirectional cross-view attention mechanism is used to integrate the syntactic structure information from the graph-enhanced representation into the sequence-enhanced representation, achieving information interaction and enhancement between the two views. The alternating dual-view fusion blocks form skip knowledge connections through residual connections to alleviate oversmoothing, allowing the representations of the two views to mutually enhance each other during the progressive deepening process. Finally, pooling and output are performed.

[0143] Phase 1: Text Encoding and Position Encoding. BiLSTM or BERT is used to extract basic semantic representations, while part-of-speech embeddings and relative distance embeddings are integrated as structural position encodings to enhance the model's ability to perceive the position of syntactic structures.

[0144] The second stage: Alternating dual-view fusion. The model stacks L alternating dual-view fusion blocks. Within each block, the following operations are executed sequentially: graph convolution based on dense message passing to mine the syntactic dependencies between aspect words and viewpoint words in the syntactic graph view; sequence transformation operations to capture long-distance semantic associations in the semantic sequence view; and unidirectional cross-view attention operations to achieve information interaction and enhancement between the two types of views. Residual connections are used between the alternating dual-view fusion blocks to form skip knowledge connections, thus mitigating oversmoothing.

[0145] The third stage: pooling and classification. Aspect-specific pooling is performed on the final output representation of the target aspect words, followed by a fully connected layer and a Softmax function to output the sentiment classification result.

[0146] This embodiment will detail the design and implementation of each module of the model:

[0147] The theoretical foundation of dependency syntax:

[0148] Dependency parsing is a core task in natural language processing (NLP) for revealing the grammatical skeleton of sentences. Its core objective is to accurately characterize the dominance and subordination relationships between words. Compared to the hierarchical division of sentence components in phrase structure grammar, dependency grammar emphasizes the direct association between words. Through the binary asymmetric relationship of "headword-dependency word," it abstracts the sentence structure into a rooted dependency grammar tree. This tree-like structure uses the core predicate or noun in the sentence as the root node, and the remaining words as child nodes. Directed edges connect these nodes to form a hierarchical system, which can intuitively and accurately reconstruct the modification, dominance, and logical relationships between language components, providing structured support for downstream semantic understanding tasks. Table 1 shows some basic dependency relationships.

[0149] Taking the sentence "The writer published an inspiring novel last month" as an example, the dependency syntax tree of this sentence clearly shows the hierarchical dependency relationships between words. In this sentence, the core verb "published" is the root node of the dependency syntax tree, governing the grammatical structure of the entire sentence. "writer" is the logical subject of "published," and the two form a subject-verb dependency relationship; "an inspiring novel" is the direct object of "published," clarifying the object of the action. The indefinite article "an" and the adjective "inspiring" both act as modifiers depending on the noun "novel," while "novel" directly depends on the core verb "published." "Last month" is a prepositional phrase indicating time, acting as an adverbial of "published," used to describe the time when the action occurred. The preposition "last" directly depends on "published," and the noun "month" depends on the preposition "last."

[0150] Table 1 Partial Dependency Model

[0151]

[0152] Applications of Dependency Syntax in Sentiment Analysis: In aspect-level sentiment analysis tasks, dependency syntax trees provide crucial structured information. By analyzing the dependency relationships between words in a sentence, the semantic associations between aspect words and their related viewpoint words can be accurately identified, which is of great significance for solving fine-grained judgments of sentiment polarity. The applications of dependency syntax are mainly reflected in the following aspects:

[0153] Aspect-Opinion Pair Extraction: Dependency parsing can effectively identify the modification relationship between aspect words and opinion words. For example, in the sentence "The camera quality is excellent but the battery drains quickly," dependency parsing can establish dependency chains of "camera" → "quality" → "excellent" and "battery" → "drains" → "quickly," thus accurately associating aspect words with their corresponding opinion words.

[0154] Feature representation of dependency paths: On a dependency syntax tree, the shortest path connecting aspect words and opinion words contains important semantic information. The word sequence and dependency relation type on this path together constitute key clues for determining sentiment polarity. For example, positive sentiment is often conveyed through relations such as "amod" (adjective modification) or "nsubj" (noun subject), while negative sentiment is often accompanied by "neg" (negative modification).

[0155] Input Construction for Graph Neural Networks: Dependency parse trees can be directly transformed into graph structures, serving as input to graph neural networks (GCN / GAT). In this representation, words act as nodes, dependency relations as edges, and edges store the dependency relation types. Through the information transfer mechanism of graph neural networks, the model can aggregate information along grammatical relation paths, enhancing the context-awareness of aspect word representations.

[0156] Analysis of oversmoothing problems in deep models: Figure 3 Examples of oversmoothing and fusion of different models: (a) GCN oversmoothing example (b) Transformer oversmoothing example (c) GCN & Transformer alternating fusion example

[0157] Deep neural networks can extract more abstract semantic features by stacking multiple layers. However, as the number of network layers increases, the model often suffers from over-smoothing. Over-smoothing, initially a major concern in graph neural network research, refers to the phenomenon where node representations gradually converge as the number of convolutional layers in a graph increases, losing local discriminative information and leading to a decline in model performance. In aspect-level sentiment analysis tasks, over-smoothing makes it difficult to distinguish node representations corresponding to different aspect words, thus weakening the model's ability to recognize fine-grained sentiment.

[0158] Graph neural networks model text as graph-structured data, constructing deep architectures through cascading multiple graph convolutional layers, such as... Figure 3 As shown in (a), graph convolution can essentially be viewed as a local smoothing operation on node features. As network depth increases, this smoothing effect gradually amplifies, causing the representation vectors of different nodes to become increasingly uniform and lose their discriminative power—this is the so-called "oversmoothing" phenomenon. In natural language processing scenarios, oversmoothing becomes a key factor hindering the expansion of GNNs to deeper layers, thus significantly constraining the improvement of model performance.

[0159] Deep Transformer models also face the risk of representation homogenization. The core of the Transformer is its self-attention mechanism, which captures global dependencies by calculating attention weights between any two positions in a sequence. As shown in the research by Shi et al., as the number of layers increases, the Transformer exhibits oversmoothing, such as... Figure 3 As shown in (b).

[0160] Basic Idea: Aspect-based sentiment analysis (ABSA) is a method for predicting the sentiment polarity of various aspects within a sentence. Compared to traditional text classification methods, ABSA is a fine-grained learning task. In recent years, due to ABSA's ability to model the connections between words in a sentence, graph neural networks have been increasingly used in natural language processing tasks, and research on ABSA tasks has also emerged. However, most studies utilizing graph convolutions are prone to oversmoothing. Furthermore, while some existing work focuses on using GNNs and Transformers to process text, methods that tightly integrate the syntactic graph view and semantic sequence view of the text remain to be explored. To address these issues, this paper proposes a sentiment analysis method based on co-modeling of syntactic graphs and semantic sequences. This method processes the syntactic graph view process using graph convolutional layers and the semantic sequence view process using Transformer layers, tightly coupling the two and mitigating the oversmoothing problem.

[0161] BERT: Originally proposed by Google, BERT's core idea is to learn general language representations through large-scale unsupervised pre-training and fine-tune them in downstream tasks, thereby significantly improving the model's semantic understanding capabilities. Because BERT can generate high-quality contextualized representations, it has demonstrated excellent performance in multiple natural language processing tasks such as sentiment analysis, text classification, and sequence labeling, and has had a significant impact on the development of subsequent pre-trained models.

[0162] From a model structure perspective, BERT is based on the Transformer Encoder and utilizes a multi-head self-attention mechanism to model the input sequence in parallel. Unlike RNNs and LSTMs, which rely on recursive computation across time steps, the Transformer can directly capture the dependencies between any two positions, thus more effectively modeling long-distance semantic relationships. During the pre-training phase, BERT primarily employs two tasks: Masked Language Model (MLM) and NextSentence Prediction (NSP), and is trained using massive amounts of corpus data. This allows the model to learn word-level semantics and contextual information, as well as acquire a certain ability to model inter-sentence relationships.

[0163] In terms of input representation, BERT's input vector consists of three parts: token embedding, segment embedding, and position embedding. Taking the sentence "Great food but the service was dreadful!" as an example, the sequence needs to be encoded according to the BERT format before being fed into the model: a special marker "[CLS]" is added to the beginning of the sequence to gather global semantic information, and "[SEP]" is added to the end of the sentence to indicate the sentence boundary; if the sequence length is less than the preset maximum length, it is padded with "[PAD]" to ensure that the batch input dimension is consistent.

[0164] This model uses BERT as its core encoder. In addition to its context-aware capabilities, it can automatically identify sentiment keywords and their modifiers in text due to the sentiment semantic modeling experience accumulated during pre-training. At the same time, the [CLS] tag vector output by BERT can provide sentence-level global semantic information. After being concatenated with aspect-specific representations, it can further enhance the context-aware capabilities of sentiment classification. This allows the model to maintain stable recognition accuracy when dealing with complex texts with multiple aspects and sentiment polarities. This is the core reason why it is more advantageous than traditional distributed representation methods in fine-grained sentiment analysis tasks.

[0165] The dual-view alternating fusion layer is a fundamental component consisting of three main parts: a graph convolutional layer (implementing graph convolution operations based on dense message passing), an intermediate layer, and a sequence transformation layer (implementing sequence transformation operations based on self-attention). We treat text as dual-view data: as a syntactic graph structure in one view and as a semantic temporal sequence in the other. On one hand, the graph convolutional layer processes the text in the syntactic graph view; it is an implementation of DenseGIN. In short, the graph convolutional layer consists of a message builder, aggregator, and updater (i.e., a multilayer perceptron, MLP), which will be detailed in later subsections. On the other hand, the sequence transformation layer processes the text as semantic sequence data, with its attention mechanism performing global computation on all words within the same instance. Furthermore, an intermediate layer is placed between the graph operators and the sequence operators, enabling the two modules to be compatible and adaptable, thus achieving a good integration.

[0166] The formula for the above process is expressed as follows:

[0167] ;

[0168] ;

[0169] ;

[0170] ;

[0171] in, and They are respectively after the first The and the first The word intermediate representation after alternating dual-view fusion layers (total) (blocks) and It is the first The outputs of the graph convolutional layer and sequence transform layer in the alternating dual-view fusion block. This is the output of the intermediate layer, i.e., LayerNorm followed by ReLU. Each dual-view alternating fusion layer has residual connections from beginning to end, which constitute skip knowledge connections. The alternation mechanism forces node representations to switch and update between different views, avoiding repeated local smoothing operations in a single view (such as a syntactic graph view), thereby effectively suppressing the homogenization trend of representations and alleviating oversmoothing.

[0172] DenseGIN: To adapt to text graph learning tasks and address the shortcomings of sparse batch processing implementations, we designed a dense batch graph convolution module for text, called DenseGIN. This module is based on the GINE concept and implements dense message passing for node and edge features. The adjacency matrix of the graph (including edge type) is denoted as... Node features are denoted as .

[0173] Before message passing, first, based on the adjacency matrix... Generate a mask matrix and simultaneously incorporate node features. Expanding along both rows and columns, we obtain... and .

[0174] Regarding the message passing mechanism, the message construction process is as follows:

[0175] ;

[0176] ;

[0177] in Represents edge feature embedding, This is the message matrix corresponding to the constructed edges. After filtering using a 0-1 mask matrix, it can be ensured that... The eigenvectors corresponding to all non-edges are set to zero.

[0178] Next, perform the aggregation operation:

[0179] ;

[0180] ;

[0181] in These represent the aggregation results for nodes and edges, respectively. and For update functions (such as multilayer perceptron MLP).

[0182] A key step in DenseGIN is to use a mask to filter redundant edge features, i.e., if a node... and If there is no boundary between them, then let This ensures that noise information does not affect the updating of the node or edge representation in the next layer.

[0183] Structural Position Encoding and Feature Fusion: To enhance positional awareness of graph structures, the model introduces part-of-speech (POS) embeddings and relative distance embeddings as node structural position encodings. Furthermore, the model achieves multi-level feature fusion and information preservation through inter-layer skip knowledge connections and residual structures, further improving the robustness of aspect-based sentiment classification.

[0184] Aspect-level sentiment analysis model based on syntactic graph and semantic sequence co-modeling

[0185] Dependency parsing and preprocessing: One of the core designs of this invention's model is to simultaneously model the text as a syntactic graph view and a semantic sequence view. The construction of the syntactic graph view depends on the dependency parsing of the input sentences. Therefore, we use Stanford CoreNLP as a preprocessing tool to perform dependency parsing on each input sentence to obtain its corresponding syntactic structure information.

[0186] Specifically, for a given sentence Stanford CoreNLP first performs basic processing such as word segmentation and part-of-speech tagging, and then constructs a dependency parser tree for the sentence. This tree structure represents the grammatical modification relationships between words in the form of directed edges, such as subject-verb, verb-object, and adjective modification relationships. We transform this into an adjacency matrix. , of which elements Indicator With words The adjacency matrix stores the dependency relationship types between aspects and viewpoints (or 0 if there is no dependency relationship). This matrix will serve as input to subsequent graph convolution modules to mine syntactic dependency paths between aspect words and viewpoint words on the syntactic graph view.

[0187] Furthermore, the dependency parser also outputs part-of-speech (POS) tags for each word, such as nouns, verbs, and adjectives. These POS tags contain rich grammatical role information, helping the model distinguish between aspect words (usually nouns) and opinion words (usually adjectives or verbs). Therefore, we inject POS embeddings as part of the structural position encoding, along with relative distance embeddings, into the model to enhance its ability to perceive syntactic structure.

[0188] Text Encoding and Positional Encoding: This model first transforms the original text into a vector representation rich in semantic and structural information. This layer comprises three sequential steps: text encoding, feature adaptation, and structural-positional encoding fusion.

[0189] For the input sentence First, each word is mapped to a low-dimensional vector through a word embedding layer. This invention provides two encoder options: a BiLSTM encoder and a BERT encoder.

[0190] The BiLSTM encoder uses pre-trained GloVe word vectors as the initial word embeddings for freezing. Then, the initial hidden state is generated by capturing contextual information through a bidirectional long short-term memory (BiLSTM) network. :

[0191] ;

[0192] The BERT encoder uses a pre-trained BERT model as the context encoder. Word sequences are input into BERT to obtain a context representation containing global semantics. And the [CLS] vector used for sentence-level classification:

[0193] ;

[0194] To enable encoder output To better suit the subsequent processing of alternating dual-view fusion blocks, the model introduces a lightweight feed-forward network (FFN) as a feature adaptation layer. This layer consists of two linear transformation layers and a GELU activation function, performing a non-linear mapping on the representation while maintaining dimensionality invariance.

[0195] ;

[0196] in, and Let these represent the first linear transformation matrix and the second linear transformation matrix in the feedforward network, respectively. and These represent the corresponding bias vectors.

[0197] In choosing the activation function, this invention employs GELU (Gaussian Error Linear Unit). The mathematical expression for GELU is:

[0198] ;

[0199] in GELU is the cumulative distribution function of a standard normal distribution. Compared to the traditional ReLU activation function, GELU is smooth and continuous across the entire network, and its gradient does not completely vanish in the negative region, effectively ensuring stable gradient propagation in deep networks. Simultaneously, GELU outputs a non-zero small value for negative inputs, preserving more potential semantic information, making it particularly suitable for sentiment analysis scenarios where negation words, transition words, and other semantic elements require negative values ​​to convey meaning. Furthermore, mainstream pre-trained models like BERT use GELU as their feedforward network (FFN) activation function. Using this function in the feature adaptation layer maintains consistency with the feature distribution output by BERT, reducing the risk of representation drift.

[0200] This feature adaptation layer is added only when using a BERT encoder; if a BiLSTM encoder is used, this layer is skipped. .

[0201] To inject syntactic structure information into the model, two structural position encodings are introduced after the feature adaptation layer: part-of-speech embedding and relative distance embedding. Part-of-speech embedding utilizes the part-of-speech (POS) tags output by the dependency parser to assign a learnable embedding vector to each word. The relative distance embedding calculates the relative distance between each word in the sentence and the target aspect word, and then discretizes it to map it into another set of learnable embedding vectors. .

[0202] The above two structural position codes and the adapted representation By concatenating the two representations, we obtain an enhanced representation that integrates semantic and structural information:

[0203] ;

[0204] To adapt this representation to the input dimensions of subsequent dual-view alternating fusion blocks, the model maps its dimensions through a linear projection layer:

[0205] ;

[0206] The final result This serves as the input to the first dual-view alternating fusion block. The design combines explicit syntactic structure information with task-adapted semantic representations, enabling the model to perceive syntactic roles and aspect relevance during the encoding phase, laying the foundation for subsequent dual-view collaborative learning.

[0207] The dual-view alternating fusion block is the core of this invention's model, aiming to deeply couple the syntactic graph view and semantic sequence view of the text. Each dual-view alternating fusion block consists of three sub-layers: a graph convolutional layer, a sequence transformation layer, and a cross-view attention layer, which are stacked sequentially and form skip knowledge connections through residual connections to alleviate oversmoothing. Output of a dual-view alternating fusion block And an adjacency matrix generated from the dependency syntax tree, storing dependency relation types. , No. The calculation process for each alternating dual-view fusion block is as follows.

[0208] Graph convolutional layer: First, using the adjacency matrix Input representation This invention employs dense message passing-based graph convolution operations to strengthen the syntactic dependency paths between aspect terms and viewpoint terms. The internal computation process is as follows:

[0209] Edge feature embedding: through learnable embedding layers The edge type index in the map is mapped to a dense edge feature vector:

[0210] ;

[0211] ;

[0212] Indicator With words Types of dependency relationships The corresponding learnable edge feature representation.

[0213] Message construction: incorporating node features Expand along the direction of travel Then with edge features Add and then activate via ReLU:

[0214] ;

[0215] ;

[0216] For the node Passed to node The original message.

[0217] Mask filtering: based on the adjacency matrix Generate a mask (keeping only the edges), and set the messages at non-edge locations to zero:

[0218] ;

[0219] Node message aggregation and update: for each node Aggregates data from all neighbors The message is then processed through a multi-layer sensor. The graph is updated to obtain the enhanced representation:

[0220] ;

[0221] Edge update: embedding edge features Node features extending along the row direction and node features extending along the column direction Added, through a multilayer perceptron Update:

[0222] ;

[0223] in The output graph augmentation representation of the graph convolutional layer aggregates information from syntactic neighbors, making the node representation rich in structured constraints; This is the updated edge feature representation, used by the next dual-view alternation fusion block.

[0224] Sequence transformation layer: The input is fed into a standard Transformer encoder layer, which utilizes global self-attention to capture long-range semantic dependencies. This layer computes attention weights for all positions in the sequence for each position, outputting an augmented representation of the sequence.

[0225] ;

[0226] Here, layer normalization is performed first before entering the sequence transformation layer to stabilize training.

[0227] Cross-view attention layer: To deeply fuse syntactic structure and semantic sequence information, a unidirectional cross-view attention mechanism is designed in each dual-view alternating fusion block. This mechanism uses the output of the graph convolutional layer... and the output of the sequence transformation layer As input, As a query, with As keys and values, syntactic structural information is integrated into the semantic representation through unidirectional attention, achieving synergistic enhancement of the two views without performing the reverse cross-view attention operation that uses graph-enhanced representation as the query.

[0228] Specifically, cross-view attention is applied to the output of the sequence transformation layer. As a query, the output of the graph convolutional layer The attention output is computed using the key and value as keys:

[0229] ;

[0230] in For a learnable projection matrix, attention is calculated using a scaled dot product:

[0231] ;

[0232] Residual Connections and Output: Finally, the output of the cross-view attention layer... Input with dual-view alternating fusion block The results are summed and then normalized through layers to obtain the final output of this dual-view alternating fusion block:

[0233] ;

[0234] This residual connection constitutes a skip knowledge connection, which not only alleviates the gradient vanishing problem in deep networks, but also preserves the underlying features and effectively suppresses the oversmoothing phenomenon.

[0235] By stacking Through such alternating dual-view fusion blocks, the model gradually constructs a deep representation that contains both rich syntactic structure and global semantics, providing high-quality input features for subsequent aspect-specific sentiment classification.

[0236] Pooling and classification output layer: after After processing the alternating dual-view fusion blocks, the final word-level representation of the sentence can be obtained. .

[0237] Aspect-specific representation pooling: for a given target aspect An aspect-specific representation of the target aspect term is generated by performing average pooling on the final output representations corresponding to these aspect terms. The calculation is shown in the following formula:

[0238] ;

[0239] in, Indicates the first Keywords In the final output representation The corresponding word vectors.

[0240] Sentiment classifier: The pooled aspect-specific representation is input into the classifier to predict sentiment polarity. When using BERT as the encoder, the aspect-specific representation is used to fuse sentence-level global information. The [CLS] vector output by BERT is concatenated to form the input of the classifier. As shown in the formula below:

[0241] ;

[0242] If using a BiLSTM encoder, then use directly. As input, the probability distribution of sentiment polarity (positive, negative, neutral) is finally obtained through a fully connected layer and a softmax function. The output sentiment classification result is shown in the following formula:

[0243] ;

[0244] The model minimizes the prediction distribution With real labels Cross-entropy loss between End-to-end training is performed, as shown in the following formula:

[0245] ;

[0246] During the classification phase, an adaptive pooling strategy (BERT concatenation [CLS]) was used to fuse local aspect information with global sentence semantics, enhancing the context-awareness of sentiment judgment. Through the organic combination of these modules, the entire model achieves multi-level, interactive modeling of syntax and semantics, significantly improving the performance and robustness of aspect-level sentiment analysis.

[0247] Test Implementation Examples

[0248] This embodiment will use three English datasets to evaluate the model: the SemEval 2014 Laptop (LAP14) and Restaurant (REST14) datasets, and the Twitter dataset.

[0249] The Restaurant and Laptop datasets contain canonical review texts for restaurants and laptops, respectively, to test the model's basic sentiment classification ability on typical product reviews. The Twitter dataset consists of sentences sourced from social media, characterized by short text, informal language, and significant noise. Each sample contains one aspect term, used to verify the model's adaptability and robustness to non-standard colloquial text. Aspect-level statistics for the datasets are shown in Table 2, and sentence-level statistics are shown in Table 3.

[0250] Table 2. Aspect-level statistics of the dataset

[0251]

[0252] Table 3. Sentence-level statistics of the dataset

[0253]

[0254] Experimental Setup: This experiment utilizes the PyTorch deep learning framework to build and train all models. PyTorch, with its dynamic computation graph mechanism and automatic differentiation function, provides high flexibility and debuggability for implementing complex structures such as graph convolutional networks and Transformers. PyTorch's concise programming interface and rich community resources effectively improve model iteration efficiency, laying a reliable technical foundation for subsequent experimental analysis.

[0255] Table 4 Experimental Environment Configuration

[0256]

[0257] The specific hyperparameters used in the experiment are shown in Table 5. Since the BERT model has already learned a large amount of language understanding knowledge during pre-training, a relatively small learning rate of 2e-5 was used for fine-tuning to preserve its pre-training knowledge. A larger learning rate was used in task-specific layers; this hybrid learning rate mechanism effectively improved model performance.

[0258] Table 5 Hyperparameter Settings

[0259]

[0260] Evaluation metrics: The main evaluation metrics for machine learning models include accuracy (Accuracy, Acc), precision (Precision, P), recall (Recall, R), and F1 score (F1). This invention uses accuracy and F1 score to compare experimental results. First, let's introduce the confusion matrix:

[0261] TP (True Positive): The number of correctly predicted positive samples;

[0262] TN (True Negative): The number of correctly predicted negative class samples;

[0263] FP (False Positive): The number of negative samples that were incorrectly predicted as positive.

[0264] FN (False Negative): The number of positive samples that were incorrectly predicted as negative.

[0265] Table 6 Confusion Matrix

[0266]

[0267] Accuracy is the proportion of correctly predicted samples out of the total number of samples. It is well-suited for tasks with balanced sample classes. The formula for calculating accuracy is as follows:

[0268] ;

[0269] In cases of class imbalance, where there are majority and minority classes, the model may perform well on the majority class but poorly on the minority class. Therefore, for imbalanced classes, precision can be used, defined as the proportion of correctly predicted positive samples out of all positive samples.

[0270] ;

[0271] Recall measures a model's ability to identify all true positive samples. It is defined as the proportion of samples correctly predicted as positive out of all samples that were actually positive.

[0272] ;

[0273] Since improving precision or recall will lead to a decrease in the other, the F1 score is introduced, which is defined as the harmonic mean of precision and recall:

[0274] ;

[0275] This embodiment will handle a three-class classification task, using Accuracy and F1 scores to evaluate model performance.

[0276] Benchmark Models: In the field of aspect-level sentiment analysis research, various methods have been extensively explored. To comprehensively and accurately evaluate the performance of the method proposed in this invention, we designed a systematic comparative experiment. The benchmark models cover a variety of mainstream methods, including traditional neural networks, pre-trained models, graph neural networks, and large language models. Different encoders have different requirements for the input and feature fusion methods of the model architecture. To ensure fair comparison and comprehensive verification of the effectiveness of the method, the benchmark models of this invention include both architectures using and not using BERT encoders. These include: InterGCN, R-GAT, DualGCN, SSEGCN, and AG-VSR based on graph convolutional networks; DGEDT based on a combination of graph attention and Transformer; and various variants combining BERT and graph networks (such as BERT4GCN, T-GCN+BERT, etc.).

[0277] (1) ChatGPT: A large language model method based on ChatGPT-3.5-turbo, which uses zero-shot and few-shot cue learning strategies to perform aspect-level sentiment classification tasks.

[0278] (2)LLaMa2-13b: an open-source large language model that uses general semantic representation to directly complete aspect-level sentiment classification of text under zero-shot learning settings.

[0279] (3) InterGCN: A model based on graph convolutional networks (GCN) that focuses on modeling the interaction between aspect words and context words in a sentence. It explicitly captures grammatical dependency information through graph structure to enhance the accuracy of aspect sentiment classification.

[0280] (4) R-GAT: The relation graph attention network (R-GAT) is adopted to introduce a relation-aware attention mechanism on the dependency graph, and at the same time, the dependency type is used as the edge feature to enhance the utilization of syntactic structure information.

[0281] (5) DGEDT: Based on dependency graph enhancement, the dual Transformer structure integrates dependency graph information into the self-attention mechanism of Transformer to jointly model the sequence information and grammatical structure of sentences.

[0282] (6) DualGCN: It uses a dual-channel graph convolutional network to extract features from the perspectives of dependency graph and semantic relevance graph respectively, and then fuses them through dual affine attention to fully capture the grammatical and semantic information of the sentence.

[0283] (7) SSEGCN: Propose Syntax-Semantic Enhanced Graph Convolutional Network (SSEGCN), which designs a mask matrix based on syntactic distance and combines aspect-aware attention to perform sentiment analysis using both syntactic and semantic information.

[0284] (8) AG-VSR: The model adopts attention-assisted graph and variational sentence representation (AG-VSR). It learns the variational representation of the sentence through the variational autoencoder structure and enhances the graph convolution with the help of the attention mechanism to improve the robustness of sentiment polarity prediction.

[0285] (9) BERT: Based on the pre-trained language model BERT, classification is performed directly through fine-tuning, utilizing its powerful contextual representation capabilities as a strong benchmark model.

[0286] (10) R-GAT+BERT: R-GAT is combined with BERT. BERT’s word representations are used as input, and R-GAT is used to further utilize dependency graph information to enhance the model’s perception of grammatical structure.

[0287] (11) DGEDT+BERT: Combining DGEDT and BERT, utilizing BERT's contextual embedding, and using a deeper feature fusion mechanism enhanced by dual Transformers and dependency graphs.

[0288] (12) BERT4GCN: Uses the output of the intermediate layer of BERT as the input of GCN, and further aggregates the neighbor information through graph convolution to realize the combination of pre-trained model and graph neural network.

[0289] (13) T-GCN+BERT: The combination of Type-Aware Graph Convolutional Network (T-GCN) and BERT uses the relation type matrix and adjacency matrix to calculate weights, thereby enhancing the aggregation process of node representation.

[0290] (14) DualGCN+BERT: DualGCN and BERT are integrated, and BERT's embedding representation is used to extract and fuse multi-view features through dual-channel graph convolution.

[0291] (15) SSEGCN+BERT: Based on SSEGCN, BERT word representation is introduced to further improve the ability to extract syntactic and semantic features and enhance the performance of sentiment classification.

[0292] (16) AG-VSR+BERT: The combination of AG-VSR and BERT uses the contextual representation of BERT as input, and then performs sentiment analysis through attention-assisted graphs and variational sentence representations.

[0293] Experimental Results: This embodiment compares the model proposed in this invention with several benchmark models on three datasets: Restaurant, Laptop, and Twitter. The experimental results are shown in Table 8. The Twitter dataset contains only sentences from a single perspective, while the Restaurant and Laptop datasets contain sentences from multiple perspectives. In the table, bold indicates the best result, underline indicates the second-best result, and "—" indicates that relevant experimental data was not provided in the original paper.

[0294] Table 7 Experimental results of the model on different datasets

[0295]

[0296] Compared with methods based on large language models, ChatGPT and LLaMa2-13b still show significant performance gaps in fine-grained sentiment analysis tasks under zero-shot and few-shot settings. The best-performing ChatGPT few-shot model achieves an accuracy of 84.62% and an F1 score of 76.08% on the Restaurant dataset, lower than our Ours model. On the Laptop dataset, its F1 score reaches 75.79%, better than the Ours model without BERT, but still lower than the Ours+BERT model. LLaMA2-13b performs even worse, with significant differences in both accuracy and F1 score. This indicates that although large language models possess powerful general semantic understanding capabilities, they are still less stable and accurate than our model in supervised, highly structured, and strongly constrained fine-grained sentiment analysis tasks.

[0297] Compared to models without BERT, the Ours model proposed in this invention achieves state-of-the-art performance on the Restaurant and Laptop datasets, which contain sentences with multiple aspects. On the Restaurant dataset, the Ours model achieves an accuracy (Accumulation) of 85.02% and an F1 score of 79.66%, representing improvements of 0.75 and 1.58 percentage points respectively compared to the suboptimal DualGCN model. On the Laptop dataset, the Ours model achieves an accuracy of 78.91% and an F1 score of 74.88%, representing improvements of 0.43 percentage points (accuracy vs. DualGCN) and 0.11 percentage points (F1 score vs. AG-VSR) respectively compared to the suboptimal models. On the Twitter dataset, which contains sentences with only one aspect, the Ours model achieves an accuracy of 77.17% and an F1 score of 74.93%, representing improvements of 1.25 and 0.60 percentage points respectively compared to the suboptimal DualGCN and AG-VSR models. The above results demonstrate that, regardless of whether the sentence is about a single aspect or multiple aspects, the model of this invention can effectively learn contextual sentiment cues related to aspect words, exhibiting stable and excellent performance.

[0298] Compared to models incorporating BERT, the proposed Ours+BERT model demonstrates further performance improvements across all datasets after introducing BERT as the base encoder. On the Restaurant dataset, the Ours+BERT model achieves an accuracy of 87.29% and an F1 score of 81.93%, representing improvements of 0.16 and 0.58 percentage points respectively compared to the suboptimal DualGCN+BERT and R-GAT+BERT models. On the Laptop dataset, the Ours+BERT model achieves an F1 score of 78.46%, a 0.36 percentage point improvement over DualGCN+BERT. On the Twitter dataset, the Ours+BERT model achieves an F1 score of 76.38%, showing a slight advantage of 0.02 percentage points over SSEGCN+BERT. These results validate the good compatibility and synergistic effect between BERT's powerful contextual representation capabilities and the model structure of this invention.

[0299] This invention's model achieves a deep fusion of syntactic structure and semantic sequence by alternately coupling dependency graph convolution and Transformer: the graph convolution module strengthens the syntactic dependency paths between aspect words and opinion words, while the Transformer module effectively captures long-distance semantic associations. This alternating stacking design alleviates the oversmoothing problem that easily occurs in single GNN or Transformer models, making the deep representation more stable and discriminative. Therefore, this invention's model achieves consistent and superior performance in both single-aspect and multi-aspect sentence scenarios.

[0300] Overall, the model's performance is significantly improved across both multi-aspect sentence datasets and single-aspect datasets. The model employs an alternating coupling structure of dependency graph convolution and Transformer: graph convolution strengthens the syntactic dependency paths between aspects and opinions, while Transformer captures long-distance semantic relationships. This alternation effectively alleviates the oversmoothing problem of pure GNNs or pure Transformers, making deep representations more stable and discriminative. Therefore, regardless of whether a sentence contains only one aspect or multiple aspects, the model can consistently learn aspect-related contextual sentiment cues, resulting in consistent and strong performance.

[0301] Ablation Experiments: To further verify the effectiveness of each core module in the proposed dual-view alternating fusion model, we conducted systematic ablation experiments on the Restaurant and Laptop datasets. We used BiLSTM as the Text Encoder and designed five model variants to examine the impact of the sequence transformation module, graph convolution module, alternating structure, cross-view attention mechanism, and module execution order on aspect-level sentiment classification performance. The specific settings for each variant are as follows:

[0302] The first group (T Only w / o Cross) retains only the sequence transformation module, completely removing the graph convolution module and cross-view attention. This variant is used to verify the necessity of the graph convolution module introducing syntactic structural information.

[0303] The second group (G Only w / o Cross) retains only the graph convolution module, completely removing the sequence transformation module and cross-view attention. This variant is used to verify the necessity of the sequence transformation module capturing long-range semantic dependencies.

[0304] The third group (w / o Cross): retains the alternating stacking structure of the graph convolution module and the sequence transformation module (in the order of graph convolution → sequence transformation), but removes the cross-view attention layer within the alternating dual-view fusion block. This variant is used to verify the contribution of the cross-view attention mechanism itself.

[0305] The fourth group (T→G w / cross): retains the complete cross-view attention mechanism, but swaps the execution order within the dual-view alternating fusion block to sequence transformation → graph convolution, i.e., semantic sequence view modeling is performed first, followed by syntactic graph view modeling. This variant is used to examine the impact of view processing order on model performance.

[0306] Group 5 (Ours): The complete model of this invention, this variant serving as an upper bound for performance.

[0307] Table 8 Ablation experimental results on different datasets

[0308]

[0309] From the ablation experiment results in Table 8, firstly, the models that retain only a single view (T Only and G Only) performed significantly worse than the full model on both datasets. Neither of them could fully extract fine-grained sentiment information in the text when used alone, which verifies the necessity of dual-view collaboration.

[0310] Comparing w / o Cross with T Only and G Only reveals that even with the removal of cross-view attention, the alternating stacking structure of graph convolutional and sequence transformation modules still delivers significant performance improvements. On the Restaurant dataset, w / o Cross improves accuracy and F1 score by 3.92% and 3.89% respectively compared to T Only, and by 4.51% and 4.95% respectively compared to G Only; similar improvements are observed on the Laptop dataset. This indicates that simple alternating stacking can effectively integrate syntactic structure and semantic sequence information, laying the foundation for dual-view collaboration.

[0311] Further comparison of Ours and w / o Cross reveals that the introduction of cross-view attention mechanism further enhances model performance. On the Restaurant dataset, Ours achieves an accuracy of 85.02% and an F1 score of 79.66%, representing improvements of 0.98% and 2.32% respectively compared to w / o Cross; on the Laptop dataset, improvements of 1.89% and 2.22% are achieved. This result directly demonstrates that cross-view attention can mutually enhance the representations of the syntactic graph view and the semantic sequence view through bidirectional interaction, thereby more accurately locating the association between aspect words and opinion words and improving the robustness of sentiment polarity discrimination.

[0312] Finally, comparing Ours and T→G w / Cross reveals that the execution order within the alternating dual-view fusion block significantly impacts model performance. On both datasets, performing graph convolution operations first outperforms performing sequence transformation operations first. On the Restaurant dataset, Ours achieves 2.27% higher accuracy and 1.90% higher F1 score than T→G w / Cross; on the Laptop dataset, it achieves 2.23% and 3.21% higher accuracy and F1 score, respectively. This indicates that the pattern of first strengthening local syntactic dependencies through graph convolution modules, then capturing global semantics with sequence transformation modules, and finally fusing cross-view information through cross-view attention is more consistent with the cognitive patterns of sentiment analysis.

[0313] In summary, the ablation experiments fully demonstrate the rationality of the design of each module in the dual-view alternating fusion model: the graph convolution module provides the necessary syntactic structural constraints for the model, the sequence transformation module captures global semantic associations, and the cross-view attention mechanism realizes deep interaction between views, effectively improving the accuracy and robustness of aspect-level sentiment analysis.

[0314] Model efficiency analysis: To measure the actual deployment performance of the model, this embodiment quantitatively compares the proposed graph convolution and sequence transformation alternating stacked structure with the pure sequence transformation structure from two dimensions: parameter quantity and inference speed. All experiments were conducted under the same hardware and hyperparameter environment. Inference speed was calculated using the full dataset of the test set, with the average single-sample inference time (ms) as the evaluation index. The results are shown in Table 9.

[0315] Table 9. Efficiency Comparison of Different Model Structures

[0316]

[0317] As shown in the table, the pure sequence transformation structure, by removing the graph convolution module and the cross-view attention module, has fewer parameters and faster inference speed. In contrast, the model of this invention uses an alternating stacking structure and adds a cross-attention mechanism, with a total parameter count increase of only 0.15M, a trainable parameter count increase of 148.6K, and an average inference time increase of only 0.16ms, resulting in a limited increase in computational cost. Combined with the experimental results in Section 3.7.5, it can be seen that the model of this invention introduces dual-view collaborative modeling capability to the model with minimal computational cost, thereby achieving significant performance improvements on datasets such as Restaurant and Laptop, achieving a balance between accuracy and efficiency.

[0318] The above results demonstrate that the syntactic graph and semantic sequence alternation fusion mechanism designed in this invention is efficient and lightweight. While improving the fine-grained sentiment analysis effect, it does not introduce too much computational burden, and has good computational efficiency and practical application potential.

[0319] Hyperparameter Setting Experiment: In aspect-level sentiment analysis models based on the alternating dual-view fusion mechanism, the number of layers in the alternating dual-view fusion block has a significant impact on model performance. To evaluate the model's sensitivity to layer depth, we conducted hyperparameter analysis experiments on the Restaurant dataset, setting the number of alternating dual-view fusion block layers from 1 to 6, while keeping other training settings consistent.

[0320] Model accuracy varies with the number of alternating fusion block layers of dual views, as follows: Figure 4 As shown in the results, the accuracy is highest and the model performs best when the number of layers is 4. However, when the number of layers is greater than 4, the accuracy decreases with increasing layer count. This is because increasing the number of layers leads to an increase in the number of model parameters and complexity, making model training more difficult. With 4 layers, the number of parameters and complexity are moderate, and the model is more efficient in feature extraction. In summary, the number of layers in the dual-view alternating fusion block in this invention is set to 4.

[0321] Case Study: To further verify the effectiveness of the proposed model in complex contexts, this embodiment selects four complex sentence samples from a public dataset for case analysis, and the results are shown in Table 10. These sentences all contain complex linguistic phenomena such as multiple aspects, contrastive relationships, and long-distance dependencies, which can effectively test the model's ability to distinguish fine-grained sentiment. It can be seen that DGEDT and DualGCN are prone to sentiment misjudgment in sentences with multiple aspects, especially when multiple polarities are intertwined, making it difficult to accurately match aspects with corresponding sentiments. For example, the first sentence contains three aspects with different sentiments, and the comparative models all misjudged neutrality and polarity; while the model of this invention, through the co-modeling of syntactic structure and semantic sequence, can accurately distinguish the sentiment polarity of different aspects and make all predictions correctly. In sentences containing contrast and long-distance dependencies, the benchmark model often misses viewpoints or confuses polarities, while the method of this invention, relying on the dual-view alternating fusion mechanism, can stably capture the dependency relationship between aspects and sentiment words, showing stronger robustness. The above examples demonstrate that the model of this invention has a more accurate and stable emotional understanding ability compared to existing methods in challenging scenarios such as complex sentence structures, multifaceted emotions, and long-distance dependencies.

[0322] Table 10 Case Analysis

[0323]

[0324] For complex structures containing multiple parallel sub-aspects nested within a main aspect, neither DGEDT nor DualGCN can accurately identify the multi-level aspect relationships, resulting in overall misjudgments. The model in this invention effectively alleviates this problem through dual-view collaborative modeling, but still exhibits a small degree of bias in densely nested scenarios, indicating that complex nested structures remain a common challenge in fine-grained sentiment analysis.

[0325] In summary, this invention focuses on aspect-level sentiment analysis tasks and addresses the shortcomings of existing methods in modeling syntactic structure and semantic sequence by proposing an aspect-level sentiment analysis method based on a dual-view alternating fusion mechanism. This method simultaneously models text as a syntactic graph view and a semantic sequence view, achieving deep coupling and collaborative learning between the two types of views through dual-view alternating fusion blocks. In model design, firstly, BiLSTM or BERT is used for basic semantic encoding, combined with part-of-speech embedding and relative distance embedding as structural position encoding to enhance position awareness. Then, multiple stacked dual-view alternating fusion blocks sequentially and alternately execute graph convolution operations based on dense message passing and sequence transformation operations based on self-attention. Dense message passing is implemented in the graph convolution layer to mine syntactic dependencies, while long-distance semantic associations are captured in the sequence transformation layer. Further feature fusion is achieved through a unidirectional cross-view attention mechanism. Finally, aspect-specific representation pooling and a sentiment classifier are used to predict sentiment polarity and output the sentiment classification result.

[0326] Experimental results on multiple public datasets, including Restaurant, Laptop, and Twitter, demonstrate that the proposed model outperforms mainstream benchmark models in both accuracy and F1 score, with a particularly significant advantage in complex sentences containing multiple aspects. Ablation experiments validate the contributions of the graph convolution module, sequence transformation module, and alternating dual-view structure to model performance, confirming the effectiveness of collaborative modeling. Model efficiency analysis confirms that the proposed method achieves significant performance improvements with minimal computational overhead and parameter increments, balancing effectiveness and inference efficiency. Hyperparameter analysis experiments show that the model achieves optimal performance when the number of alternating dual-view fusion block layers is set to 4. Typical case studies further intuitively verify the model's robustness in complex sentence structures and scenarios with intertwined sentiment polarities. The aspect-level sentiment analysis method proposed in this invention, based on collaborative modeling of syntactic graphs and semantic sequences, effectively alleviates the oversmoothing problem in traditional models by deeply fusing the syntactic structure and semantic information of the text, improving the model's ability to discriminate fine-grained sentiment and providing an effective solution for aspect-level sentiment analysis in complex scenarios.

[0327] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. An aspect-level sentiment analysis method based on syntactic graphs and semantic sequence modeling, executed by a computer, for predicting the sentiment polarity of a given target aspect word in an input sentence, characterized in that... Includes the following steps: Text encoding and position encoding steps: Perform basic semantic encoding on the input sentence to obtain the initial semantic representation of the input sentence; Based on the dependency parsing tree obtained from the dependency parsing of the input sentence and the relative positions of the target aspect words, generate structural position encoding, which includes part-of-speech embedding and relative distance embedding; The initial semantic representation is fused with the structural position encoding to generate a fused semantic representation; The alternating dual-view fusion step involves using the fusion semantic representation as initial input and processing it sequentially through multiple alternately stacked dual-view fusion blocks. Each dual-view fusion block performs operations in the following order: First, on the syntactic graph view, based on the adjacency matrix constructed from the dependency syntax tree and storing dependency relation types, a graph convolution operation based on dense message passing is performed on the input representation to aggregate the neighbor node information that has a dependency relation with the current node, thereby obtaining a graph-enhanced representation; Secondly, on the semantic sequence view, a self-attention-based sequence transformation operation is performed on the input representation to capture global semantic dependencies and obtain a sequence-enhanced representation; Finally, a one-way cross-view attention operation is performed, using the sequence augmented representation as the query and the graph augmented representation as the key and value. The syntactic structure information in the graph augmented representation is integrated into the sequence augmented representation through the attention mechanism to generate a cross-view fusion representation, without performing the reverse cross-view attention operation that uses the graph augmented representation as the query. The cross-view fusion representation is residually connected to the input representation of the dual-view alternating fusion block. This residual connection constitutes a jump knowledge connection from the block input to the block output to alleviate oversmoothing, thereby obtaining the output representation of the dual-view alternating fusion block. Sentiment classification output steps: Based on the final output representation of the alternating stacked dual-view alternating fusion blocks, extract the aspect-specific representation of the target aspect word, and predict the sentiment polarity of the target aspect word accordingly.

2. The aspect-level sentiment analysis method based on syntactic graphs and semantic sequence modeling as described in claim 1, characterized in that, The text encoding and position encoding steps specifically include: The input sentence is context-encoded using the pre-trained language model BERT to obtain the initial semantic representation and a classification tag vector CLS located at the beginning of the sentence to represent the global semantics at the sentence level.

3. The aspect-level sentiment analysis method based on syntactic graphs and semantic sequence modeling as described in claim 2, characterized in that, Before generating the fused semantic representation, the process also includes: The initial semantic representation is input into a feature adaptation layer for processing. The feature adaptation layer consists of two linear transformation layers and a GELU activation function located between them.

4. The aspect-level sentiment analysis method based on syntactic graphs and semantic sequence modeling according to claim 2, characterized in that, In the text encoding and position encoding steps, the method for generating the fused semantic representation is as follows: The initial semantic representation, the part-of-speech embedding, and the relative distance embedding are concatenated, and then dimensional mapping is performed through a linear projection layer to obtain the fused semantic representation. The part-of-speech embedding is obtained by mapping part-of-speech tags obtained by performing dependency parsing on the input sentence; the relative distance embedding is obtained by calculating the relative distance between each word in the input sentence and the target aspect word, and then mapping it after discretization.

5. The aspect-level sentiment analysis method based on syntactic graphs and semantic sequence modeling as described in claim 3, characterized in that, In the text encoding and position encoding steps, the method for generating the fused semantic representation is as follows: The semantic representation processed by the feature adaptation layer, the part-of-speech embedding, and the relative distance embedding are concatenated, and then dimensional mapping is performed through a linear projection layer to obtain the fused semantic representation. The part-of-speech embedding is obtained by mapping part-of-speech tags obtained by performing dependency parsing on the input sentence; the relative distance embedding is obtained by calculating the relative distance between each word in the input sentence and the target aspect word, and then mapping it after discretization.

6. The aspect-level sentiment analysis method based on syntactic graphs and semantic sequence modeling according to claim 1, characterized in that, The graph convolution operation based on dense message passing is specifically a text-oriented dense batch graph convolution operation, including: Based on the adjacency matrix, edge feature embeddings are generated for the edges that represent dependency relationship types. The input representation is expanded along the row direction, added to the edge feature embedding, and activated by ReLU to obtain the original message matrix; Generate a mask based on the adjacency matrix and set the messages at non-edge positions in the original message matrix to zero; For the message matrix after zeroing, for each node, aggregate the messages of all its neighbors along the neighbor dimension to obtain the aggregated representation of the node, and perform a nonlinear transformation through a node update function implemented by a multilayer perceptron to obtain the graph augmented representation; Furthermore, the edge feature embedding is added to the node features extended along the row direction and the node features extended along the column direction, and then subjected to a nonlinear transformation through an edge update function implemented by a multilayer perceptron to obtain the updated edge feature representation.

7. The aspect-level sentiment analysis method based on syntactic graphs and semantic sequence modeling according to claim 1, characterized in that, In the unidirectional cross-view attention operation, the sequence augmentation representation and the graph augmentation representation are mapped to queries, keys, and values ​​by multiplying with their respective learnable projection matrices, and attention weights are calculated.

8. The aspect-level sentiment analysis method based on syntactic graphs and semantic sequence modeling according to claim 1, characterized in that, The sentiment classification output step specifically includes: Average pooling is performed on the final output representation of each word contained in the target aspect word to obtain the aspect-specific representation of the target aspect word; The aspect-specific representation is input into a classifier to predict the sentiment polarity probability distribution of the target aspect word.

9. The aspect-level sentiment analysis method based on syntactic graphs and semantic sequence modeling according to claim 2, characterized in that, The sentiment classification output step specifically includes: Average pooling is performed on the final output representation of each word contained in the target aspect word to obtain the aspect-specific representation of the target aspect word; The aspect-specific representation is concatenated with the classification label vector CLS corresponding to the input sentence to obtain the classification input vector. The classification input vector is fed into a fully connected layer and a Softmax function to predict the sentiment polarity probability distribution of the target aspect words.

10. An aspect-level sentiment analysis system based on collaborative modeling of syntactic graphs and semantic sequences, characterized in that, include: Memory, used to store the computer programs required to build aspect-level sentiment analysis models; A processor, configured to execute computer programs stored in the memory to enable the operation of various modules of the aspect-level sentiment analysis model, including: The text encoding and position encoding module includes: a text encoder for semantic encoding of the input sentence; and a position encoding fusion layer for fusing structural position encodings generated based on a dependency syntax tree to output a fused semantic representation. Multiple alternately stacked dual-view alternating fusion modules, each of the dual-view alternating fusion modules comprising: The graph convolutional layer based on dense message passing is used to aggregate syntactic neighbor information on the graph view. It includes an edge feature embedding unit, a message construction unit, a mask filtering unit, a node message aggregation unit, a node update unit, and an edge update unit. A self-attention-based sequence transformation layer is used to capture global semantic dependencies on the sequence view; A cross-view attention layer is used to perform unidirectional attention calculations using the output of the sequence transformation layer as a query and the output of the graph convolution layer as a key and value to achieve information fusion between views. The residual connection unit is used to add the output of the cross-view attention layer to the input representation of the dual-view alternating fusion module by residual addition to form a jump knowledge connection to alleviate oversmoothing. The pooling and sentiment classification module includes: an aspect-specific representation pooling layer for extracting aspect-specific representations of target aspect words from the output of the last dual-view alternating fusion module; and a sentiment classifier for predicting sentiment polarity based on the aspect-specific representations.

Images