An aspect-level sentiment analysis method based on the fusion of pre-trained dependency trees and dynamic knowledge graphs
By generating dependency trees and constructing dynamic knowledge graphs through pre-trained language models, and combining them with multi-head attention graph convolutional networks, this approach addresses the instability of dependency parsing and insufficient fusion of external knowledge in existing methods for colloquial and noisy texts, achieving aspect-level sentiment analysis with high accuracy and interpretability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-02-27
- Publication Date
- 2026-06-02
AI Technical Summary
Existing aspect-level sentiment analysis methods lack robustness in dependency parsing on colloquial and noisy texts. The sparsity of the dependency matrix limits the expression of long-range dependencies. They also lack a systematic introduction of external domain knowledge, making it difficult to identify implicit aspects and implicit sentiments, and their ability to disambiguate polysemous words is insufficient.
Dependency trees are generated and corrected by pre-trained language models, a dynamic knowledge graph is constructed and aligned with the syntactic graph across graphs, and a multi-head attention graph convolutional network is used to achieve joint modeling of syntax and knowledge, thereby improving the robustness and interpretability of sentiment analysis.
It improves the accuracy and stability of emotion recognition in complex scenarios, enhances the ability to recognize implicit aspects and polysemous words, and supports the rapid adaptation and optimization of the model under domain evolution.
Smart Images
Figure CN122133642A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of natural language processing (NLP) technology and relates to an aspect-level sentiment analysis method based on the fusion of pre-trained dependency trees and dynamic knowledge graphs. Background Technology
[0002] With the explosive growth of user-generated content, aspect-level sentiment analysis (ABSA) is becoming increasingly important in scenarios such as public opinion monitoring, product feedback, and service optimization. Existing methods mostly rely on the combination of syntactic dependency trees and deep semantic representations, but traditional dependency parsing is not robust enough in colloquial and noisy text; the sparsity of dependency matrices limits the expression of long-range dependencies; at the same time, the lack of systematic introduction of external domain knowledge makes it difficult to fully identify implicit aspects and implicit sentiments, and is also not conducive to disambiguation of polysemous words.
[0003] Recent studies have attempted to enhance aspect-context interactions using graph convolutional networks (GCNs) or attention mechanisms, but problems remain: (1) unstable dependency structure quality; (2) coarse granularity of external knowledge and syntactic information fusion; and (3) insufficient adaptability in different domains / scenarios. Therefore, it is necessary to propose a novel ABSA method that simultaneously addresses the fusion of syntactic quality improvement and knowledge enhancement to improve robustness and interpretability in complex scenarios. Summary of the Invention
[0004] In view of this, the purpose of this invention is to provide an aspect-level sentiment analysis method based on the fusion of pre-trained dependency trees and dynamic knowledge graphs. By directly generating / correcting dependency trees through a pre-trained language model, a robust syntactic graph is obtained. At the same time, entity-relation triples are extracted using a large language model (LLM) to construct a dynamic knowledge graph and align and fuse it with the syntactic graph across graphs. Finally, a multi-head attention graph convolutional network is used to achieve joint modeling of "syntax × knowledge", thereby improving the accuracy and stability of implicit aspect and sentiment polarity recognition.
[0005] To achieve the above objectives, the present invention provides the following technical solution: An aspect-level sentiment analysis method based on the fusion of pre-trained dependency trees and dynamic knowledge graphs, comprising: Acquire text corpora in the target domain, annotate them by sentence-aspect pairs, and construct a dataset; Dependency relations are extracted from the target text sequence using a pre-trained language model to obtain a candidate dependency tree. At the same time, a baseline dependency tree is obtained by extracting dependency relations from the same target text sequence using a traditional dependency parser. The structural consistency between the candidate dependency tree and the baseline dependency tree is evaluated. When the structural consistency is lower than a set threshold, a backoff / weighted fusion strategy is triggered to form a stable syntactic graph. A large language model is invoked to extract <entity / term, relation, entity / term> triples from the target sentences in the target text sequence. Noisy relations are removed by threshold filtering and conflict resolution. Synonym / aliasing is then performed on multi-source entities and aligned with vector similarity. A domain knowledge graph is then instantiated. Using aspect terms and proper nouns as anchors, word-entity mapping and context anchor alignment are performed to establish cross-graph correspondence between syntactic graphs and knowledge graphs; Graph convolution features are extracted on the syntactic graph and knowledge graph respectively, and cross-graph information interaction and weighted fusion are performed through multi-head attention to obtain aspect-oriented representations; The aspect-oriented representation is input into the classifier to predict and output aspect-level sentiment polarity.
[0006] Furthermore, dependency extraction is performed using a pre-trained language model to obtain candidate dependency trees. and will The adjacency matrix is denoted as Dependency relation extraction is performed using a traditional dependency parser to obtain the baseline dependency tree. and will The adjacency matrix is denoted as ; The structural consistency between candidate dependency trees and baseline dependency trees is evaluated using the intersection-union ratio of edge sets.
[0007] In the formula, For structural consistency score, Let the set of edges of the candidate dependency tree be . Let be the set of edges of the baseline dependency tree; Set a threshold ,when Time Adjacency Matrix and Perform weighted fusion:
[0008] In the formula, This is the merged syntactic adjacency matrix. These are the weight coefficients for the pre-trained dependency tree; make This yields the syntax diagram:
[0009] in, To add the adjacency matrix after adding self-loops, It is the identity matrix. For matrix The degree matrix, for The value of the element in the i-th row and j-th column. For the set of nodes in the syntax graph, This is the normalized symmetric adjacency matrix.
[0010] Furthermore, a large language model is invoked to extract <entity / term, relation, entity / term> triples from the target sentence and its context expansion window, resulting in knowledge triples. and confidence level , For entities and entity The relationship between them; Calculate the overall relevance: , These are the weighting coefficients for semantic similarity. To extract the weighting coefficients for confidence levels, For entities The vector representation of , A vector representation of the context; when Retain triples , For the threshold; When there are conflicting relationships between the same entity pairs, the highest confidence source is retained first; if the confidence is close, the cosine similarity between the semantic vector of the conflicting relationship and the context vector is calculated, and the relationship category with higher semantic similarity to the context is retained. Merging is performed based on a thesaurus / alias dictionary and vector similarity to achieve multi-entity alignment; then, the knowledge graph is instantiated using the filtered triples. , It is the set of entity nodes in the knowledge graph.
[0011] Furthermore, by using aspect terms and proper nouns as anchors, an initial mapping between syntactic graph nodes and knowledge graph entities is established. During alignment, window constraints are used to limit the search range, and alignment confidence is assigned to the mapping. Set a threshold to apply a mask to mappings with alignment confidence less than the threshold; additionally, apply a mask to unaligned mappings.
[0012] Furthermore, graph convolutions are stacked on the syntactic graph and the knowledge graph respectively to extract structural features; then, using one graph as the query matrix and the other graph as the key matrix and value matrix, a multi-head attention mechanism is used for cross-graph fusion; aggregate pooling is performed on aspect word positions to obtain aspect-oriented representations; this aspect-oriented representation is input into a classifier to predict and output aspect-level sentiment polarity.
[0013] Furthermore, the method also includes training a classifier using cross-entropy loss.
[0014] Furthermore, in addition to the cross-entropy loss, alignment consistency loss and L2 regularization loss can be combined to train the classifier, and the total loss is expressed as:
[0015] In the formula, The weighting coefficients for the alignment consistency loss. These are the weighting coefficients for the L2 regularization loss. For cross-entropy loss, For alignment consistency loss, This is the L2 regularized loss.
[0016] Furthermore, the method also includes recording the knowledge graph version number during knowledge graph construction, and updating only the knowledge graph when domain terminology evolves. The version does not require retraining of the main model, which is a neural network structure other than the dynamic knowledge graph.
[0017] The beneficial effects of this invention are as follows: (1) This invention generates and corrects dependency trees through a pre-trained language model and introduces a consistency evaluation and weighted fusion mechanism to improve the robustness and accuracy of syntactic parsing. It can adaptively reconcile the results of the pre-trained model and the traditional parser, overcome the problem that the syntactic structure in non-standard text is easily affected by noise, and generate a stable and reliable syntactic graph, thereby providing a structural basis for aspect-level sentiment analysis and enhancing the model's adaptability in colloquial or noisy scenarios.
[0018] (2) This invention integrates external domain knowledge into the sentiment analysis process by dynamically constructing a knowledge graph and implementing cross-graph alignment. This invention uses a large language model to extract entity relation triples and filters and resolves conflicts to ensure the accuracy and consistency of knowledge. It not only strengthens the recognition of explicit aspect words, but also captures implicit aspects and implicit sentiments, solving the problems of polysemous word disambiguation and missing contextual semantics.
[0019] (3) This invention employs a multi-head attention graph convolutional network to integrate syntactic information with external knowledge. It integrates the two information sources through cross-graph attention interaction and weighted integration, and combines residual connections and layer normalization to ensure training stability. This improves the overall accuracy of aspect extraction and opinion polarity discrimination, enhances the interpretability of the model, and makes the decision-making process more transparent. In addition, this invention supports online hot updates of the knowledge graph by recording version numbers, enabling the model to quickly adapt to domain evolution and maintain high performance and continuous optimization capabilities in high-noise scenarios.
[0020] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description
[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the aspect-level sentiment analysis method based on the fusion of pre-trained dependency tree and dynamic knowledge graph provided in an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating the principle of the aspect-level sentiment analysis method described in this invention. Detailed Implementation
[0022] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0023] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0024] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.
[0025] Please see Figure 1and Figure 2 One embodiment of the present invention provides an aspect-level sentiment analysis method based on the fusion of pre-trained dependency trees and dynamic knowledge graphs, as described below: 1. Data preparation and annotation, the specific implementation process is as follows: (1) Corpus collection: Select texts from e-commerce reviews / social media comments, clean and remove duplicates, and retain sentences with a length of 5 to 120 words.
[0026] (2) Sample construction: Organize samples according to “sentence-aspect”. If multiple aspects appear in a sentence, split it into multiple training samples.
[0027] (3) Labeling specifications: Aspects are labeled using BIO sequence labels or explicitly given; polarity categories (positive, negative, neutral).
[0028] (4) Segmentation ratio: The training / validation / test set is divided in an 8:1:1 ratio; oversampling or loss reweighting is used for long-tail categories.
[0029] 2. Based on the pre-trained language model fine-tuned by the task, dependency relation extraction is performed to obtain candidate dependency trees; and the structural consistency is evaluated with the results of the traditional parser. When the result is lower than the set threshold, a backoff / weighted fusion strategy is triggered to form a stable syntactic graph (adjacency representation with self-loops and normalization).
[0030] The specific implementation process is as follows: (1) Dependency generation: Based on a pre-trained language model (such as BERT / RoBERTa) fine-tuned for the task domain, dependency relations are extracted from the target text sequence to obtain candidate dependency trees. Its adjacency matrix is denoted as .
[0031] (2) Baseline parsing: The baseline dependency tree is obtained by calling a traditional dependency parser on the same target text sequence. The adjacency matrix is .
[0032] (3) Consistency assessment: Structural consistency is characterized by the intersection and union ratio of edge sets.
[0033] in, For structural consistency score, Let the set of edges of the candidate dependency tree be . Let be the set of edges of the baseline dependency tree. When ( As a confidence threshold, it is recommended Fusion is triggered when ( ).
[0034] (4) Rollback / fusion: Weighted fusion is used to form stable dependencies:
[0035] in, This is the merged syntactic adjacency matrix. The weighting coefficients for the pre-trained dependency tree are in the range of [0,1]. Adaptive search is possible on the validation set.
[0036] (5) Normalized graph construction: Let This yields the syntax diagram:
[0037] in, To add the adjacency matrix after adding self-loops, It is the identity matrix. For matrix The degree matrix, For matrix The value of the element in the i-th row and j-th column. For the set of nodes in the syntax graph, This is the normalized symmetric adjacency matrix.
[0038] 3. Use a large language model to extract <entity / term, relation, entity / term> triples, and assign confidence and relevance scores to each relation; remove noisy relations through threshold filtering, blacklist rules and conflict resolution; perform synonym / alias merging and vector similarity alignment for multi-source entities, instantiate the domain knowledge graph Gk, and support versioned incremental updates.
[0039] The specific implementation process is as follows: (1) Triple Extraction: A large language model (LLM) is invoked to extract entities / terms and relations from the target sentences and their context expansion windows in the target text sequence, resulting in... and confidence level .in, For entities i and entity j The relationship between them.
[0040] (2) Relevance Calculation and Filtering: Define the overall relevance:
[0041] in, These are the weighting coefficients for semantic similarity. To extract the weighting coefficients for confidence levels, For entities The vector representation of , A vector representation of the context. When ( recommend When retaining the candidate triplet, .
[0042] (3) Conflict resolution: If there is a conflict relationship between the same entity pair, the highest confidence source is retained first; if the confidence is close, the context consistency is used to decide. Specifically, the cosine similarity between the semantic vector of the conflict relationship and the context vector is calculated, and the relationship category with higher semantic similarity to the context is retained.
[0043] (4) Entity alignment: Based on the thesaurus / alias dictionary and vector similarity, multiple entity alignment is completed. Specifically, a threshold is set. (e.g., 0.8), calculate the cosine similarity between the vector representations of different entity references. If the similarity is higher than a set threshold, the similarity is considered. If the entity is not aligned, it will be treated as the same entity and its ID will be merged and disambiguated. Noisy entities that fail to align will be removed.
[0044] (5) Graph instantiation and versioning: Constructing knowledge graphs:
[0045] in, This is a collection of entity nodes in the knowledge graph. A version number v is recorded to support incremental updates.
[0046] 4. Establish using terms and proper nouns as anchor points. Nodes and The mapping relationship of entities; when aligning, window constraints are used to limit the search range and alignment confidence is assigned to the mapping; attention masks are applied to misaligned or insufficient confidence positions to constrain the weight distribution of subsequent cross-graph attention.
[0047] The specific implementation process is as follows: (1) Anchor point selection: Set the aspect terms and proper nouns as anchor points. ,Establish Nodes and Initial mapping of entities.
[0048] (2) Window constraint: Set a window w (e.g., 5 words on the left and right) in the sentence for each anchor point to limit the candidate entities.
[0049] (3) Mapping and confidence: Establishing the mapping matrix And calculate the alignment confidence. .
[0050] (4) Attention mask: Apply an attention mask to misaligned and insufficiently confident mappings, such as to ( For the threshold, a value of [value] is recommended. The mapping of the graph is masked to block invalid cross-graph connections, thereby constraining the weight distribution of subsequent cross-graph attention.
[0051] 5. In respectively and Graph convolution features are extracted, and cross-graph information interaction and weighted fusion are performed through multi-head attention to obtain aspect-oriented representations. These representations are then input into a classifier to predict and output aspect-level sentiment polarity.
[0052] The specific implementation process is as follows: (1) In-image feature extraction: In and Stacking 1-2 layers of GCN on each layer extracts local and global structural features, resulting in a feature representation of the syntactic graph. and the feature representation of knowledge graphs .
[0053] (2) Cross-graph attention interaction: in and In the middle, one of the images is used as the query matrix. The other diagram is the key matrix. Sum matrix Employing multi-head attention (number of heads) Perform cross-graph fusion:
[0054] The cross-graph fusion output is stabilized through residual connections and layer normalization training.
[0055] (3) Aspect aggregation: Aggregate pooling is performed on the position of aspect words or their mask range to obtain aspect-oriented fused representation. .
[0056] (4) Polarity classification: The fused representation is input into the fully connected layer, and the polarity probability distribution is obtained through the Softmax function:
[0057] in, This is the weight matrix of the fully connected layer. This is the bias vector.
[0058] 6. Training, inference, and online updates: The specific implementation process is as follows: (1) Loss function: The main loss is cross-entropy:
[0059] in, For the true label of category c, The predicted probability for category c. Index for sentiment polarity categories.
[0060] Optionally, an alignment consistency loss can also be added. (For constraint diagrams) and Semantic consistency at anchor points and L2 regularization The total loss can then be expressed as:
[0061] in, The weighting coefficients for the alignment consistency loss. The weighting coefficients for L2 regularization loss. This is the set of all learnable parameters for the model.
[0062] (2) Optimization and strategy: AdamW optimizer, learning rate batch is set to Training rounds Early stopping and gradient clipping are used (e.g., 1.0).
[0063] (3) Reasoning process: For the input sentence and aspect, execute S2 (syntactic analysis) -> S3 (triple extraction and knowledge graph construction) -> S4 (cross-graph alignment) -> S5 (feature fusion and classification) in sequence, and directly output the predicted polarity; cache the high-frequency entity mapping and alignment results to reduce latency and computational overhead.
[0064] (4) Online hot update: When domain terminology evolves, only the knowledge graph is updated. version No retraining of the main model is required. The main model refers to the neural network structure other than the dynamic knowledge graph, which mainly includes a pre-trained language model encoder, a graph convolutional network (GCN) for the syntactic graph and knowledge graph, a cross-graph attention fusion module, and a sentiment classifier. The new version of the triplet will be implemented in a phased rollout and then fully switched to after evaluation on the validation set.
[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. An aspect-level sentiment analysis method based on the fusion of pre-trained dependency trees and dynamic knowledge graphs, characterized in that, The method includes: Acquire text corpora in the target domain, annotate them by sentence-aspect pairs, and construct a dataset; Dependency relations are extracted from the target text sequence using a pre-trained language model to obtain a candidate dependency tree. At the same time, a baseline dependency tree is obtained by extracting dependency relations from the same target text sequence using a traditional dependency parser. The structural consistency between the candidate dependency tree and the baseline dependency tree is evaluated. When the structural consistency is lower than a set threshold, a backoff / weighted fusion strategy is triggered to form a stable syntactic graph. A large language model is invoked to extract <entity / term, relation, entity / term> triples from the target sentences in the target text sequence. Noisy relations are removed by threshold filtering and conflict resolution. Synonym / aliasing is then performed on multi-source entities and aligned with vector similarity. A domain knowledge graph is then instantiated. Using aspect terms and proper nouns as anchors, word-entity mapping and context anchor alignment are performed to establish cross-graph correspondence between syntactic graphs and knowledge graphs; Graph convolution features are extracted on the syntactic graph and knowledge graph respectively, and cross-graph information interaction and weighted fusion are performed through multi-head attention to obtain aspect-oriented representations; The aspect-oriented representation is input into the classifier to predict and output aspect-level sentiment polarity.
2. The method according to claim 1, characterized in that, Dependency extraction is performed using a pre-trained language model to obtain candidate dependency trees. and will The adjacency matrix is denoted as ; Dependency relation extraction is performed using a traditional dependency parser to obtain the baseline dependency tree. and will The adjacency matrix is denoted as ; The structural consistency between candidate dependency trees and baseline dependency trees is evaluated using the intersection-union ratio of edge sets. In the formula, For structural consistency score, Let the set of edges of the candidate dependency tree be . Let be the set of edges of the baseline dependency tree; Set a threshold ,when Time Adjacency Matrix and Perform weighted fusion: In the formula, This is the merged syntactic adjacency matrix. These are the weight coefficients for the pre-trained dependency tree; make This yields the syntax diagram: in, To add the adjacency matrix after adding self-loops, It is the identity matrix. for The degree matrix, For matrix The element in the i-th row and j-th column, For the set of nodes in the syntax graph, It is the normalized symmetric adjacency matrix.
3. The method according to claim 1, characterized in that, The large language model is invoked to extract <entity / term, relation, entity / term> triples from the target sentence and its context expansion window, thus obtaining knowledge triples. and confidence level , For entities and entity The relationship between them; Calculate the overall relevance: , These are the weighting coefficients for semantic similarity. To extract the weighting coefficients for confidence levels, For entities The vector representation of , A vector representation of the context; when Retain triples , For the threshold; When there are conflicting relationships between the same entity pairs, the highest confidence source is retained first; if the confidence is close, the cosine similarity between the semantic vector of the conflicting relationship and the context vector is calculated, and the relationship category with higher semantic similarity to the context is retained. Merging is performed based on a thesaurus / alias dictionary and vector similarity to achieve multi-entity alignment; then, the knowledge graph is instantiated using the filtered triples. , It is the set of entity nodes in the knowledge graph.
4. The method according to claim 1, characterized in that, Using aspect terms and proper nouns as anchors, an initial mapping between syntactic graph nodes and knowledge graph entities is established. During alignment, window constraints are used to limit the search range, and alignment confidence is assigned to the mapping. Set a threshold and apply a mask to mappings with alignment confidence less than the threshold; In addition, a mask is applied to the misaligned mapping.
5. The method according to claim 1, characterized in that, Graph convolutions are stacked on the syntactic graph and the knowledge graph respectively to extract structural features; then, using one graph as the query matrix and the other graph as the key matrix and value matrix, a multi-head attention mechanism is used for cross-graph fusion; aggregate pooling is performed on aspect word positions to obtain aspect-oriented representations; this aspect-oriented representation is input into a classifier to predict and output aspect-level sentiment polarity.
6. The method according to claim 1, characterized in that, The method also includes training a classifier using cross-entropy loss.
7. The method according to claim 6, characterized in that, Based on the cross-entropy loss, the classifier is trained by combining alignment consistency loss and L2 regularization loss. The total loss is expressed as: In the formula, The weighting coefficients for the alignment consistency loss. These are the weighting coefficients for the regularization loss. For cross-entropy loss, To compensate for the alignment consistency loss, This is the L2 regularized loss.
8. The method according to claim 1, characterized in that, The method also includes recording the knowledge graph version number during knowledge graph construction, and updating only the knowledge graph as domain terminology evolves. The version does not require retraining of the main model, which is a neural network structure other than the dynamic knowledge graph.