Fine-grained aspect-level sentiment analysis method based on double syntax information
By constructing phrase trees and dependency trees, utilizing a hierarchically stacked graph attention block network, and combining aspect-context graphs, the problems of insufficient syntactic structure noise and relationship capture in traditional methods are solved, achieving higher accuracy in aspect-level sentiment analysis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-10
- Publication Date
- 2026-04-07
AI Technical Summary
Traditional aspect-level sentiment analysis methods rely on predefined rules or artificial features, which are difficult to effectively handle complex texts. Furthermore, the reliance on tree structures introduces noise and fails to capture complex syntactic relationships and inter-aspect relationships.
We adopt a dual-syntactic information-based approach, which constructs phrase trees and dependency trees, utilizes a hierarchically stacked graph attention block network to encode syntactic structure information, and constructs an aspect-context graph to fuse emotional feature representations within and between contexts.
It improves the accuracy of aspect-level sentiment analysis, effectively capturing emotional information within and between contexts, and enhances the accuracy of fine-grained aspect-level sentiment analysis.
Smart Images
Figure CN115879439B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to natural language processing technology, and more particularly to a fine-grained aspect-level sentiment analysis method based on bisyntactic information. Background Technology
[0002] Traditional aspect-level sentiment analysis methods typically rely on predefined linguistic rules or manually extracted features, making them ill-suited for complex texts. Recently, many studies have relied on deep networks to automatically extract sentiment features. Based on the assumption that "words closer to the target aspect are more relevant to its sentiment," some works have used recurrent neural networks or convolutional neural networks combined with attention mechanisms to extract sequential features; however, this assumption is not always valid. To address this issue, many studies have explored how to effectively utilize non-sequential features, such as using graph neural networks to encode the syntactic information of dependency trees. However, the inherent tree structure of dependency trees can introduce noise signals, such as unrelated connections between clauses. This noise is detrimental to modeling the sentiment perception context of a single aspect (i.e., contextual sentiment features). Moreover, the dependency tree structure only reflects the relationships between words, and therefore, in most cases, it cannot capture the complex relationships present in a sentence (e.g., sequential, causal, and adversative relationships), thus failing to model inter-aspect relationships (i.e., contextual sentiment features). Therefore, relying solely on the syntactic structure information of dependency trees has significant limitations. Summary of the Invention
[0003] The technical problem to be solved by the present invention is to provide a fine-grained aspect-level sentiment analysis method based on dual syntactic information, addressing the deficiencies in the prior art.
[0004] The technical solution adopted by this invention to solve its technical problem is: a fine-grained aspect-level sentiment analysis method based on dual syntactic information, comprising:
[0005] 1) Model the contextual information of emotion perception for each aspect and generate corresponding context-specific emotion representations;
[0006] 1.1) Given the input text {w1, w2, ..., w n} and aspect word a t Using BERT to generate context-aware word representations:
[0007]
[0008] Where, n′, m′ t These represent the lengths of the input text and aspect words after BERT word segmentation, respectively.
[0009] The context representation of the corresponding word is obtained by using the average sub-word representation.
[0010]
[0011] Among them, BertT(w i ) indicates that w is returned. i The set of coordinates of the divided sub-words in the BERT sequence, where || represents the size of the returned set;
[0012] That is, aspect word a t The corresponding context-aware word representation is as follows Where i = 1, 2, ..., n;
[0013] 1.2) Given the input text {w1, w2, ..., w n The phrase tree syntactic structure information and the dependency tree syntactic structure information are encoded by constructing a hierarchically stacked graph attention block (HGATBlock);
[0014] 1.2.1) Graph construction;
[0015] Each level of the phrase tree contains multiple phrases {ph} u Each phrase represents an independent semantic unit, following a bottom-up logic. The corresponding adjacency matrix CA is constructed as follows:
[0016]
[0017] Where l represents the l-th level of the phrase tree, This represents the set of phrases at level l of the phrase tree.
[0018] Dependency trees represent the connections between words, and the corresponding adjacency matrix (DA) is constructed as follows:
[0019]
[0020] 1.2.2) Double syntax perception.
[0021] Fusion of phrase-level syntactic information and clause-level dependency structure information;
[0022] First, based on the syntactic structure of the phrase tree, we divide it into clauses and remove all dependency edges across clauses. Then, based on the remaining dependency edges, for each level of the phrase tree, we simultaneously consider aggregating information from words within the same phrase and words with direct dependencies.
[0023] The corresponding adjacency matrix FA is formally represented as follows:
[0024]
[0025] in, This represents the operation of adding elements to the corresponding positions of the condition; that is, first setting the position corresponding to the cross-clause edge in DA to 0, and then performing CA. l Add an operation at the corresponding position of DA;
[0026] 1.2.3) Syntactic encoder.
[0027] The above hierarchical syntactic information is encoded using a hierarchically stacked graph attention block (HGAT Block) network structure, and the process can be formally expressed as follows:
[0028]
[0029]
[0030]
[0031] Where, N l (i) represents the node w at level l. i The neighborhood group, Indicator w i The final representation of at layer l; FC represents a fully connected feedforward network. It is in the l-th layer w i The representation after processing by the masked self-attention mechanism, where || represents vector concatenation, Z represents the number of attention heads, and σ is the activation function. is the trainable model parameter of the z-th attention head in the l-th layer, and f represents the scoring function that measures the relevance between two words;
[0032] Multiple HGAT Blocks are applied to the adjacency matrix FA described above, enabling the word representation of the input text to incorporate rich phrase syntax information and dependency syntax information; for convenience, the representation of the output of the last layer is defined as follows. Therefore, the aspect word a t The representation after the above syntactic structure encoding is as follows:
[0033] 1.3) By using vector addition, the output representation of the context module includes both context-aware sequence information and hierarchical syntactic structure information; that is, for aspect a... t The corresponding contextually specific emotional characteristics are expressed as follows:
[0034]
[0035] 2) Enhanced emotional feature representation based on inter-contextual relationships in modeling;
[0036] First, we propose a mapping function to find keywords (called phrase segmentation terms) that reflect the relationship between two aspects. Then, we construct an aspect-context graph based on the returned phrase segmentation terms and all aspects to model the relationship between aspects.
[0037] 2.1) Phrase Segmentation. We observe that the relationship between aspects can be reflected by phrase segmentation terms such as conjunctions. Therefore, we design a rule-based mapping function PS that returns the corresponding phrase segmentation terms for two given aspect words.
[0038] The PS process works as follows: Given two aspect words, first find their least common ancestor (LCA) node in the phrase tree. This node contains information about both aspects and has the least amount of irrelevant context. We call the branch in the subtree of the LCA that lies between the subtrees containing the two aspects the "middle branch". If the middle branch exists, return all text words on the middle branch; otherwise, return all words between the two aspects in the input text sequence. The above process can be formally represented as follows:
[0039]
[0040] Where Br(a) i a j Return to aspect a i a j This corresponds to all the text words on the middle branch.
[0041] 2.2) Aspect - Context Graph Construction.
[0042] We observe that the influence range of an aspect should be continuous, and the mutual influence between aspects decays with distance. Considering the relationships between all aspect pairs would introduce noise over long distances and increase the computational load; therefore, we only model the relationships between neighboring aspects. Figure 4 As shown, after extracting the corresponding phrase segmentation terms for each neighbor pair using the PS mapping function (e.g.) Figure 2 (As shown in red), we construct an aspect-context graph by connecting the corresponding aspect words and phrase segmentation terms to help deduce the relationships between aspects. The aspect-context graph allows each aspect word to aggregate neighbor information under the influence of its corresponding phrase segmentation term. Simultaneously, considering the bidirectional influence relationships in the aspect-context graph, we construct two corresponding adjacency matrices. The first matrix handles the influence of odd-order aspect words on adjacent even-order aspect words in all sequence aspect words, while the second handles the reverse influence. We apply the hierarchically stacked graph attention blocks (HGATBlock) constructed above as a relation encoder to the aspect-context graph, for each aspect a... t Output emotional expressions that enhance the relationship between contexts.
[0043] 3) Integrating context-specific emotional feature vectors Emotional feature vectors that enhance inter-contextual relationships Obtain the overall emotional feature vector of this aspect.
[0044]
[0045] Then, the overall sentiment features are mapped to a low-dimensional semantic space, and the softmax function is used for normalization to obtain the final prediction result.
[0046] p(t) = softmax(W p o t +b p )
[0047] Among them W p b p These are the trainable parameters of the emotion classifier model.
[0048] The overall loss function is defined as follows:
[0049]
[0050] Here, y(t) represents aspect a. t The one-hot encoded vector of the true emotion label, loss represents the standard cross-entropy, and θ represents the model training parameters.
[0051] The beneficial effects of this invention are:
[0052] 1. This invention utilizes two types of syntactic structure information from phrase trees and dependency trees to construct corresponding phrase graphs and dependency graphs, and proposes a hierarchically stacked graph attention network block structure, enabling aspect word representation to hierarchically integrate phrase-level short syntax information and clause-level dependency syntax information;
[0053] 2. This invention proposes an aspect-context graph construction method based on phrase syntax structure, which enables each aspect word to aggregate neighbor aspect information under the influence of corresponding phrase segmentation terms reflecting the relationship between aspects, thereby modeling a sentiment feature representation that enhances the relationship between contexts.
[0054] 3. This invention effectively utilizes the syntactic information of phrase trees and dependency trees to capture aspect-specific emotional content (empath-specific emotional feature representation within the context) and inter-aspect relational emotional content (empathic feature representation enhanced by inter-contextual relations), and jointly models the emotional feature representation within the context and between contexts, thereby improving the accuracy of aspect-level emotion classification and thus enhancing the accuracy of fine-grained aspect-level emotion analysis. Attached Figure Description
[0055] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:
[0056] Figure 1 This is a schematic diagram of the fine-grained aspect-level sentiment analysis method according to an embodiment of the present invention;
[0057] Figure 2 This is a schematic diagram of the phrase tree structure according to an embodiment of the present invention;
[0058] Figure 3 This is a schematic diagram of the structure of a hierarchically stacked graph attention block encoder according to an embodiment of the present invention;
[0059] Figure 4 This is a schematic diagram of the modeling contextual feature aspect-context graph construction method according to an embodiment of the present invention. Detailed Implementation
[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0061] like Figure 1 As shown, a fine-grained aspect-level sentiment analysis method based on dual syntactic information includes:
[0062] 1) Model the contextual information of emotion perception for each aspect and generate corresponding context-specific emotion representations;
[0063] 1.1) Given the input text {w1, w2, ..., w n} and aspect word a t We utilize BERT to generate context-aware word representations; we treat multi-word aspects as words, a t That is, the t-th word in the text sequence;
[0064] First, we use BERT to generate context-aware word representations;
[0065] Through the construction of BERT sequences, which are emphasized
[0066]
[0067] Obtain the output word representation.
[0068]
[0069] Where, n′, m′ t These represent the lengths of the input text and aspect words after BERT segmentation, respectively. Considering the BERT segmentation mechanism, {w iThe word is divided into multiple subwords, so we obtain the context representation of the corresponding word through the average subword representation.
[0070]
[0071] Among them, BertT(w i Return w i The set of coordinates of the divided sub-words in the BERT sequence, || returns the size of the set; therefore, aspect word a t The corresponding context-aware word representation is as follows
[0072] 1.2) Given the input text {w1, w2, ..., w n The phrase tree syntactic structure information and the dependency tree syntactic structure information are encoded by constructing a hierarchically stacked graph attention block (HGATBlock);
[0073] 1.2.1) Graph construction;
[0074] Each level of the phrase tree contains multiple phrases {ph} u}, and each phrase represents an independent semantic unit, such as Figure 2 As shown; following a bottom-up logic, the corresponding adjacency matrix CA is constructed as follows:
[0075]
[0076] Where l represents the l-th level of the phrase tree, This represents the set of phrases at level l of the phrase tree.
[0077] Dependency trees represent the connections between words, and the corresponding adjacency matrix (DA) is constructed as follows:
[0078]
[0079] 1.2.2) Double syntax perception.
[0080] Fusion of phrase-level syntactic information and clause-level dependency structure information;
[0081] First, based on the syntactic structure of the phrase tree, we divide it into clauses and remove all dependency edges across clauses. Then, based on the remaining dependency edges, for each level of the phrase tree, we simultaneously consider aggregating information from words within the same phrase and words with direct dependencies.
[0082] The corresponding adjacency matrix FA is formally represented as follows:
[0083]
[0084] in, This represents the operation of adding elements to the corresponding positions of the condition; that is, first setting the position corresponding to the cross-clause edge in DA to 0, and then performing CA. l Add an operation at the corresponding position of DA;
[0085] 1.2.3) Syntactic encoder.
[0086] like Figure 3 As shown, a hierarchical stacked graph attention block (HGAT Block) network structure is used to encode the aforementioned hierarchical syntactic information.
[0087] The HGAT Block comprises multiple graph attention layers, employing a masked self-attention mechanism to aggregate information from neighbors, and utilizing a fully connected feedforward network to map feature representations to a common semantic space. The attention mechanism enables it to handle the specificity between neighbors, assigning higher weights to more relevant words;
[0088] The process can be formally described as follows:
[0089]
[0090]
[0091]
[0092] Where, N l (i) represents the node w at level l. i The neighborhood group, Indicator w i The final representation of at layer l; FC represents a fully connected feedforward network. It is in the l-th layer w i The representation after processing by the masked self-attention mechanism, where || represents vector concatenation, Z represents the number of attention heads, and σ is the activation function. is the trainable model parameter of the z-th attention head in the l-th layer, and f represents the scoring function that measures the relevance between two words;
[0093] Multiple HGAT blocks are applied to the adjacency matrix FA described above, enabling the word representation of the input text to incorporate rich phrase syntax information and dependency syntax information. For convenience, the representation of the output of the last layer is defined as follows: Therefore, the aspect word a t The representation after the above syntactic structure encoding is as follows:
[0094] 1.3) By using vector addition, the output representation of the context module includes both context-aware sequence information and hierarchical syntactic structure information; that is, for aspect a... tThe corresponding contextually specific emotional characteristics are expressed as follows:
[0095]
[0096] 2) Enhanced emotional feature representation based on inter-contextual relationships in modeling;
[0097] First, we propose a mapping function to find keywords (called phrase segmentation terms) that reflect the relationship between two aspects. Then, we construct an aspect-context graph based on the returned phrase segmentation terms and all aspects to model the relationship between aspects.
[0098] 2.1) Phrase Segmentation. We observe that the relationship between aspects can be reflected by phrase segmentation terms such as conjunctions. Therefore, we design a rule-based mapping function PS that returns the corresponding phrase segmentation terms for two given aspect words.
[0099] The PS process works as follows: Given two aspect words, first find their least common ancestor (LCA) node in the phrase tree. This node contains information about both aspects and has the least amount of irrelevant context. We call the branch in the subtree of the LCA that lies between the subtrees containing the two aspects the "middle branch". If the middle branch exists, return all text words on the middle branch; otherwise, return all words between the two aspects in the input text sequence. This process can be formally represented as follows:
[0100]
[0101] Where Br(a) i ,a j Return to aspect a i a j This corresponds to all the text words on the middle branch.
[0102] 2.2) Aspect - Context Graph Construction.
[0103] We observe that the influence range of an aspect should be continuous, and the mutual influence between aspects decays with distance. Considering the relationships between all aspect pairs would introduce noise over long distances and increase the computational load; therefore, we only model the relationships between neighboring aspects. Figure 4 As shown, after extracting the corresponding phrase segmentation terms for each neighbor pair using the PS mapping function (e.g.) Figure 2(As shown in red), we construct an aspect-context graph by connecting the corresponding aspect words and phrase segmentation terms to help deduce the relationships between aspects. The aspect-context graph allows each aspect word to aggregate neighbor information under the influence of its corresponding phrase segmentation term. Simultaneously, considering the bidirectional influence relationships in the aspect-context graph, we construct two corresponding adjacency matrices. The first matrix handles the influence of odd-order aspect words on adjacent even-order aspect words in all sequence aspect words, while the second handles the reverse influence. We apply the hierarchically stacked graph attention blocks (HGATBlock) constructed above as a relation encoder to the aspect-context graph, for each aspect a... t Output emotional expressions that enhance the relationship between contexts.
[0104] 3) Integrating context-specific emotional feature vectors Emotional feature vectors that enhance inter-contextual relationships Obtain the overall emotional feature vector of this aspect.
[0105]
[0106] Then, the overall sentiment features are mapped to a low-dimensional semantic space, and the softmax function is used for normalization to obtain the final prediction result.
[0107] p(t) = softmax(W p o t +b p )
[0108] Among them W p b p These are the trainable parameters of the emotion classifier model.
[0109] The overall loss function is defined as follows:
[0110]
[0111] Here, y(t) represents aspect a. t The one-hot encoded vector of the true emotion label, loss represents the standard cross-entropy, and θ represents the model training parameters.
[0112] Experiments show that the aspect-level sentiment analysis method of this invention achieves better results compared to existing mainstream methods. The experiments were evaluated using four benchmark aspect-level sentiment classification datasets: the SemEval2014 (Task 4) Laptop and Restaurant datasets; the MAMS dataset; and the Twitter dataset. Detailed information about the corpora is shown in Table 1.
[0113] Table 1 Corpus Information
[0114]
[0115] The experimental section aims to evaluate the effectiveness of the aspect-level sentiment analysis model proposed in this invention on different datasets. Specifically, we list the accuracy and standard F1 score on four datasets as evaluation metrics for the model, and the experimental comparison results are given in Table 2.
[0116] Table 2. Comparative Experiment Results of Fine-Grained Aspect-Level Sentiment Analysis
[0117]
[0118] It is noteworthy that, as observed in the two tables above, the method proposed in this invention consistently outperforms all these benchmark models on different datasets. This is because these models cannot effectively utilize syntactic structure information to simultaneously capture relevant emotional information within and between contexts. This invention, for the first time, combines phrase tree syntactic structure information and dependency tree syntactic structure information, effectively utilizing the clear phrase boundaries provided by the phrase tree to hierarchically aggregate contextual information from the bottom up. The hierarchically stacked graph attention network block structure proposed in this invention can handle different word importance levels, enabling aspect word representations to hierarchically fuse phrase-level syntactic information and clause-level dependency syntactic information, modeling aspect-specific emotional feature representations within context. Based on the phrase tree hierarchical structure, this invention constructs an aspect-context graph, which can effectively utilize the mutual influence of aspect representations under phrase separators, and utilize the correlations between aspects to model emotional feature representations enhanced by inter-contextual relationships. By modeling the emotional features of aspect words within and between contexts, the accuracy of aspect-level sentiment analysis can be improved.
[0119] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.
Claims
1. A fine-grained aspect-level sentiment analysis method based on dual-syntactic information, characterized in that, include: 1) Model the contextual information of emotion perception for each aspect and generate corresponding context-specific emotion representations; 1.1) Given the input text {w1, w2, ..., w n } and aspect word a t Generate context-aware word representations Where i = 1, 2, ..., n; that is, aspect term a t The corresponding context-aware word representation is as follows 1.2) Given the input text {w1, w2, ..., w n By constructing hierarchically stacked graph attention blocks, the fusion information of phrase tree syntactic structure information and dependency tree syntactic structure information is encoded. Where i = 1, 2, ..., n, l represents the layer number; the output of the last layer is defined as... Then aspect word a t The representation after the above syntactic structure encoding is as follows: 1.3) By using vector addition, the output representation of the context module includes both context-aware sequence information and hierarchical syntactic structure information; that is, for aspect word a... t The corresponding contextually specific emotional characteristics are expressed as follows: in, A context-specific emotional feature vector; 2) Enhanced emotional feature representation based on inter-contextual relationships in modeling; 2.1) Phrase segmentation: Using a rule-based mapping function PS, the corresponding phrase segmentation terms for two given aspect words are returned. The PS process works as follows: Given two aspect words, first find their least common ancestor (LCA) node in the phrase tree. This node contains information about both aspects and has the least irrelevant context. The branch in the subtree of the LCA that is between the subtrees containing the two aspects is called the "middle branch". If the middle branch exists, return all text words on the middle branch; otherwise, return all words between the two aspects in the input text sequence. The formal representation of the above process is as follows: Among them, Br(a i ,a j Return to aspect a i ,a j All text words corresponding to the middle branch; 2.2) Aspect-Context Graph Construction After extracting the corresponding phrase segmentation item for each neighbor aspect pair using the PS mapping function, the corresponding aspect words and phrase segmentation items are connected to construct an aspect-context graph. Construct two corresponding adjacency matrices, where the first matrix handles the influence of odd-order aspect words on adjacent even-order aspect words in all sequence aspect words, and the second matrix handles the reverse influence; The hierarchical stacked graph attention blocks constructed in step 1.2) are applied to the aspect-context graph as a relation encoder for each aspect word a. t Output emotional expressions that enhance the relationship between contexts. 3) Integrating context-specific emotional feature vectors Emotional feature vectors that enhance inter-contextual relationships Obtain the overall emotional feature vector of this aspect. The overall emotional features are then mapped to a low-dimensional semantic space to obtain the final sentiment analysis prediction results.
2. The fine-grained aspect-level sentiment analysis method based on dual-syntactic information according to claim 1, characterized in that, In step 1.1), for the given input text {w1,w2,…,w n } and aspect word a t Using BERT to generate context-aware word representations, the representations are as follows: Where, n′,m′ t These represent the lengths of the input text and aspect words after BERT word segmentation, respectively. The context representation of the corresponding word is obtained by using the average sub-word representation. Among them, BertT(w i ) indicates that w is returned. i The set of coordinates of the divided sub-words in the BERT sequence, where || represents the size of the returned set.
3. The fine-grained aspect-level sentiment analysis method based on dual-syntactic information according to claim 1, characterized in that, In step 1.2), the phrase tree syntactic structure information and dependency tree syntactic structure information are encoded by constructing a hierarchically stacked graph attention block, as follows: 1.2.1) Graph construction; Each level of the phrase tree contains multiple phrases {ph} u Each phrase represents an independent semantic unit, following a bottom-up logic, and the corresponding adjacency matrix CA is constructed as follows: Where l represents the l-th level of the phrase tree, This represents the set of phrases at level l of the phrase tree. Dependency trees represent the connections between words, and the corresponding adjacency matrix (DA) is constructed as follows: 1.2.2) Dual-syntactic perception; Fusion of phrase-level syntactic information and clause-level dependency structure information; First, based on the syntactic structure of the phrase tree, clauses are divided and all dependency edges across clauses are deleted; then, based on the remaining dependency edges, for each level of the phrase tree, information is aggregated from words in the same phrase and words with direct dependencies. The corresponding adjacency matrix FA is formally represented as follows: in, This represents the operation corresponding to the condition, that is, first setting the position corresponding to the cross-clause edge in DA to 0, and then performing CA. l Add an operation at the corresponding position of DA; 1.2.3) Syntactic Encoder The above hierarchical syntactic information is encoded using a hierarchically stacked graph attention block (HGAT Block) network structure, and the process can be formally expressed as follows: Where, N l (i) represents the node w at level l. i The neighborhood group, Indicator w i The final representation of at layer l; FC represents a fully connected feedforward network. It is in the l-th layer w i The representation after processing by the masked self-attention mechanism, where || denotes vector concatenation, Z represents the number of attention heads, and σ is the activation function. is the trainable model parameter of the z-th attention head in the l-th layer, and f represents the scoring function that measures the relevance between two words; Multiple HGAT Blocks are applied to the adjacency matrix FA mentioned above, which enables the word representation of the input text to incorporate rich short syntax information and dependency syntax information.
4. The fine-grained aspect-level sentiment analysis method based on dual-syntactic information according to claim 1, characterized in that, In step 3), the overall emotional features are mapped to a low-dimensional semantic space, and the softmax function is used for normalization to obtain the final prediction result p(t). p(t)=softmax(W p o t +b p ) Among them, W p b p These are the trainable parameters of the emotion classifier model; The overall loss function is defined as follows: Where y(t) represents aspect word a t The one-hot encoded vector of the true emotion label, loss represents the standard cross-entropy, and θ represents the model training parameters.