Aspect-level sentiment analysis method based on composition tree and multiple attention mechanisms
By combining self-attention, aspect-aware attention, and graph attention networks, an aspect-level sentiment analysis method based on composition trees and multiple attention mechanisms is proposed, which solves the problem of insufficient modeling caused by dependency trees and achieves more accurate aspect-level sentiment analysis.
Patent Information
- Application Number
- CN202411176667.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-22
AI Technical Summary
Existing aspect-level sentiment analysis methods suffer from inadequacies in modeling and alignment with sentiment-representing words due to their heavy reliance on dependency trees, resulting in insufficient accuracy.
We employ a composition tree and multi-attention mechanism approach. By constructing an aspect-level sentiment analysis network, combining self-attention, aspect-aware attention, and graph attention networks, and fusing information from composition trees and dependency trees, we utilize the BiAffine module to perform interactive modeling of syntax and semantics, generating more accurate semantic and syntactic features.
It effectively integrates syntactic structure and semantic relevance, improves the alignment accuracy of aspect words and sentiment words, and enhances the accuracy of aspect-level sentiment analysis.
Smart Images

Figure CN119005203A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of sentiment analysis, and in particular to an aspect-level sentiment analysis method based on a composition tree and a multiple attention mechanism. Background Art
[0002] In recent years, with the rapid development of Internet technology, various social media platforms and e-commerce have flourished, which not only provides a broad channel for netizens to express their views and opinions, but also brings a massive amount of comment text data with rich emotional colors. Mining people's opinions based on text has a positive effect on improving the service quality of online platforms, strengthening social public opinion monitoring and improving merchants' marketing strategies.
[0003] Sentiment analysis is a natural language processing technique that aims to identify sentiment tendencies and sentiment polarity in text. Sentiment analysis is divided into sentence-level, chapter-level, and aspect-level analysis. In traditional sentiment analysis, usually only the sentiment tendency of the entire text is focused on, while aspect-based sentiment analysis (ABSA) aims to deeply explore the sentiment tendency related to specific aspects in the text.
[0004] Current research on aspect-level sentiment analysis mainly focuses on the attention mechanism, which is, however, easily affected by noise in sentences, i.e., irrelevant words. In addition, there are also some studies that explicitly use dependency trees (i.e., the syntactic structure of sentences) to help the attention mechanism more accurately identify the interactions between aspects and opinion expressions. However, despite the success of dependency trees in some tasks, methods that rely heavily on dependency trees still have shortcomings in terms of accurate modeling and alignment with words representing sentiment, as dependency trees may introduce irrelevant noise. Summary of the invention
[0005] In view of the shortcomings of existing aspect-level sentiment analysis methods in modeling and aligning with words representing sentiments due to their heavy reliance on dependency trees, the present invention provides an aspect-level sentiment analysis method based on composition trees and multiple attention mechanisms.
[0006] The present invention provides an aspect-level sentiment analysis method based on a composition tree and a multiple attention mechanism, comprising:
[0007] Step 1: construct an aspect-level sentiment analysis network, including a feature extraction module based on a multi-attention mechanism, a BiAffine module and a sentiment analysis module; the feature extraction module includes a self-attention mechanism module, an aspect-aware attention mechanism module and a graph attention network;
[0008] Step 2: construct a training data set; each training sample in the training data set includes a training text, an aspect word in the training text, and a sentiment label corresponding to the aspect word;
[0009] Step 3: Generate an encoding vector sequence of the training text, and generate a semantic vector of the training text using a self-attention mechanism module and an aspect-aware attention mechanism module according to the encoding vector sequence; wherein each vector in the encoding vector sequence corresponds to a word in the training text;
[0010] Step 4: Generate a composition tree and a dependency tree of the training text, and fuse the information of the composition tree and the dependency tree of the training text according to the encoding vector sequence to obtain a syntactic feature matrix of the training text;
[0011] Step 5: Take each word as a node, and use the graph attention network to aggregate neighbor node information according to the syntactic feature matrix of the training text to generate a syntactic vector of the training text;
[0012] Step 6: Input the semantic vector and the syntactic vector of the training text into the BiAffine module to obtain the interactive features of the training text;
[0013] Step 7: Based on the post-interaction features, the sentiment analysis module is input to predict the sentiment polarity of each aspect of the training text, and the loss between the predicted sentiment polarity and the real sentiment label is minimized to optimize the parameters of the aspect-level sentiment analysis network to obtain the aspect-level sentiment analysis model;
[0014] Step 8: Input the target text and the aspect words in the target text into the aspect-level sentiment analysis model to obtain the sentiment polarity of the aspect words in the target text.
[0015] Furthermore, in step 3, a BERT encoder is used to generate an encoding vector sequence of the training text.
[0016] Furthermore, in step 3, according to the encoding vector sequence, a semantic vector of the training text is generated using a self-attention mechanism module and an aspect-aware attention mechanism module, specifically including:
[0017] In the self-attention mechanism module, the self-attention weight matrix A of the training text is calculated according to the following formula: self ;
[0018]
[0019] Where Q and K represent the query matrix and key matrix composed of word element vectors, d represents the dimension of word element vector, and W Q and W K The learnable weight matrices corresponding to Q and K respectively;
[0020] In the aspect-aware attention mechanism module, the aspect-aware attention weight matrix A of the training text is calculated according to the following formula: asp ;
[0021] A asp =tanh(H a W a ×(KW k ) T + b);
[0022] Among them, H a represents the aspect representation determined based on all aspect word vectors in the training text; K represents the encoding vector sequence of the training text, W a Indicates and W k Respectively represent H a The learnable weight matrix corresponding to K, b represents the learnable bias; tanh represents the activation function;
[0023] Generate the semantic vector H of the training text according to the following formula sem ;
[0024] H sem =A self +A asp .
[0025] Furthermore, in step 4, the composition tree and dependency tree information of the training text are fused according to the encoding vector sequence to obtain a syntactic feature matrix of the training text, which specifically includes:
[0026] Constructing a word-level dependency adjacency matrix DA according to the dependency tree of the target text and the encoding vector sequence;
[0027] Constructing a clause-level component adjacency matrix CA according to the component tree of the target text and the encoding vector sequence;
[0028] A syntactic feature matrix is determined according to the dependency adjacency matrix DA and the component adjacency matrix CA.
[0029] Furthermore, step 5 specifically includes:
[0030] Assume that the graph attention network consists of L graph attention layers connected in series, with the number of attention heads being Z.
[0031] Will As the input of the first layer, the relationship between node i and its neighbor node j in the lth layer is calculated according to the following formula
[0032]
[0033] in, It represents the feature vector extracted from the row of a specific word in the syntactic feature matrix. It represents the set of neighbor nodes of node i in layer l when the time step is not considered. represents the final representation of node i in the l-1 layer, f is the scoring function that measures the relevance of two words, j and j' represent different neighbor nodes in the neighbor node set, and t represents the time step;
[0034] The masked attention mechanism is used to aggregate the information of all neighbor nodes of node i according to the following formula to obtain the intermediate representation of node i in layer l:
[0035]
[0036] in, represents the learnable parameters of the zth head in the lth layer, σ represents the activation function, ∥ represents vector concatenation, It represents the set of neighbor nodes of node i in layer l when considering the time step;
[0037] The final representation of node i in the l-1 layer and the intermediate representation of node i in the l layer are mapped using a fully connected feedforward network according to the following formula to obtain the final representation of node i in the l layer;
[0038]
[0039] Among them, FC represents a fully connected feed-forward network.
[0040] Furthermore, in step 6, the post-interaction features of the training text include the post-interaction semantic vector and the post-interaction syntactic vector:
[0041] H syn′ =softmax(H sem W1(H syn ) T )H syn ;
[0042] H sem′ =softmax(H syn W2(H sem ) T )H sem ;
[0043] Among them, H syn′ and H syn Represent the syntactic vectors before and after the interaction, H sem′ and H sem They represent the semantic vectors after and before the interaction respectively, W1 and W2 represent learnable weight matrices, and softmax represents the normalized activation function.
[0044] Further, in step 7, the sentiment analysis module includes a maximum pooling layer, a multi-layer perceptron and a softmax layer; correspondingly, based on the post-interaction features input to the sentiment analysis module, the sentiment polarity of various aspects of the words in the training text is predicted, specifically including:
[0045] The interactive features are first subjected to a maximum pooling operation by a maximum pooling layer, then mapped by a multi-layer perceptron, and finally subjected to sentiment classification by a softmax layer to predict the sentiment polarity of each aspect of the words.
[0046] Furthermore, in step 7, the loss between the predicted sentiment polarity and the true sentiment label is calculated according to the following formula;
[0047]
[0048] Where D represents the training dataset, s represents the training text, a represents all aspect words in s, (s, a) represents the sentence aspect pair (s, a), C is the sentiment label set, θ represents all learnable parameters in the aspect-level sentiment analysis network, y(a) represents the true sentiment label corresponding to a, p(a) represents the predicted sentiment polarity, and loss represents the cross entropy loss.
[0049] Beneficial effects of the present invention:
[0050] (1) The present invention adopts an innovative method of syntactically enhancing graph neural networks that integrate component trees, effectively integrating the syntactic structure and semantic relevance of the ABSA task. The graph attention network (GAT) is used to encode the grammatical information of the component trees in ABSA. The attention weights between nodes are dynamically learned through multiple GAT layers, and the aggregation of neighbor node information is realized, so that the model can learn more about syntactic structure. The introduction of this syntactic information can help the model better understand the dependencies between different words, thereby establishing a more accurate connection between aspect words and sentiment words, solving the problem that traditional methods may lead to inaccurate alignment of sentiment words and aspect words due to the lack of full utilization of syntactic structure.
[0051] (2) The present invention uses the BiAffine module to model between syntax and semantics to capture the semantic relationship and syntactic structure between words in a sentence, thereby promoting information transfer between different modules. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 An aspect-level sentiment analysis network framework constructed in an aspect-level sentiment analysis method based on a composition tree and a multiple attention mechanism is provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0054] In the present invention, let s = {w1, w2, ..., w n} represents a text (also called a sentence), A={a1,a2,...,a m} represents a predefined set of aspects. For each s, A s ={a i |a i ∈A and a i ∈s} represents the aspect words contained in s, which is simply recorded as a i It also represents the i-th word of s. Then the task of ABSA is to predict each aspect word a i ∈A s The emotional polarity i ∈{pos, neg, neu}, pos means positive, neg means negative, and neu means neutral.
[0055] Combination Figure 1 As shown, an embodiment of the present invention provides an aspect-level sentiment analysis method based on a composition tree and a multiple attention mechanism, comprising the following steps:
[0056] S101: constructing an aspect-level sentiment analysis network, including a feature extraction module based on a multiple attention mechanism, a BiAffine module and a sentiment analysis module; the feature extraction module includes a self-attention mechanism module, an aspect-aware attention mechanism module and a graph attention network;
[0057] S102: constructing a training data set; each training sample in the training data set includes a training text, an aspect word in the training text, and a sentiment label corresponding to the aspect word;
[0058] S103: Generate a coding vector sequence of the training text, and generate a semantic vector of the training text using a self-attention mechanism module and an aspect-aware attention mechanism module according to the coding vector sequence; wherein each vector in the coding vector sequence corresponds to a word in the training text;
[0059] Specifically, combining the self-attention mechanism enables the model to better understand the relationship between words, and using the aspect-aware attention mechanism allows the model to focus on words related to specific aspects. Therefore, this embodiment adopts the aspect-aware attention mechanism combined with the self-attention mechanism to extract semantic information related to specific aspects from the text and obtain semantic features.
[0060] S104: generating a composition tree and a dependency tree of the training text, and fusing information of the composition tree and the dependency tree of the training text according to the encoding vector sequence to obtain a syntactic feature matrix of the training text;
[0061] S105: taking each word as a node, and aggregating neighbor node information using a graph attention network according to the syntactic feature matrix of the training text to generate a syntactic vector of the training text;
[0062] S106: inputting the semantic vector and the syntactic vector of the training text into a BiAffine module to obtain the post-interaction features of the training text;
[0063] S107: inputting the post-interaction features into a sentiment analysis module, predicting the sentiment polarity of each aspect of the training text, minimizing the loss between the predicted sentiment polarity and the real sentiment label to optimize the parameters of the aspect-level sentiment analysis network, and obtaining an aspect-level sentiment analysis model;
[0064] S108: Inputting the target text and the aspect words in the target text into the aspect-level sentiment analysis model to obtain the sentiment polarity of the aspect words in the target text.
[0065] The aspect-level sentiment analysis method based on composition tree and multiple attention mechanism provided by the embodiment of the present invention adopts an innovative method of syntax-enhanced graph neural network integrating composition tree by introducing composition tree, and effectively integrates the syntactic structure and semantic relevance of ABSA task. The Graph Attention Network (GAT) is used to encode the grammatical information of the composition tree in ABSA, and the attention weights between nodes are dynamically learned through multiple GAT layers to achieve the aggregation of neighbor node information. The BiAffine module is used to model between syntax and semantics to capture the semantic relationship and syntactic structure between words in a sentence, thereby promoting information transfer between different modules.
[0066] In one embodiment, a BERT encoder is used to generate a sequence of encoding vectors for training text.
[0067] Specifically, taking the text "The drink menu is limited but all the wines are excellent" as an example, the text is input into the BERT encoder in the format of "[CLS] sentence [SEP] sentence [MASK][SEP]", that is: [CLS] The drink menu is limited but all the wines are excellent [SEP] The drink menu is [MASK][SEP]. Then, the BERT encoder captures the long-distance dependencies in the input text sequence and provides a deep representation for each word unit, so that the representation of each word unit contains the information of its context. Finally, the BERT encoder outputs the encoding vector sequence H, each vector in which corresponds to a word unit in the input text sequence and encodes the context information of the word unit.
[0068] Typically, the hidden states of the last or second-to-last layer of a BERT encoder are considered to be the most representative states.
[0069] It can be understood that the entire text is composed of sentence words (ie, non-aspect words) and aspect words, and the encoding vector sequence H of the entire text is obtained by merging or fusing sentence word encoding and aspect word encoding. The fusion method can be concatenation or weighted summation.
[0070] In one embodiment, according to the encoding vector sequence, a semantic vector of the training text is generated using a self-attention mechanism module and an aspect-aware attention mechanism module, specifically including:
[0071] In the self-attention mechanism module, the self-attention weight matrix A of the training text is calculated according to the following formula: self ;
[0072]
[0073] Where Q and K represent the query matrix and key matrix composed of word element vectors, d represents the dimension of word element vector, and W Q and W K The learnable weight matrices corresponding to Q and K respectively;
[0074] It can be understood that the advantage of the self-attention mechanism is that it can dynamically assign weights to each element according to the relationship between the elements in the input sequence, thereby obtaining a more flexible and accurate representation in different contexts.
[0075] In the aspect-aware attention mechanism module, the aspect-aware attention weight matrix A of the training text is calculated according to the following formula: asp ;
[0076] A asp =tanh(H a W a ×(KW k ) T + b);
[0077] Among them, H a represents the aspect representation determined according to all aspect word vectors in the training text (in this embodiment, the mean pooling operation is performed on the encoding vector sequence H, and the processed output is copied n times to obtain ), as the query matrix; K represents the encoding vector sequence H of the training text, as the key matrix, W a Indicates and W k Respectively represent H a The learnable weight matrix corresponding to K, b represents the learnable bias; tanh represents the activation function;
[0078] Generate the semantic vector H of the training text according to the following formula sem ;
[0079] H sem =A self +A asp .
[0080] Specifically, in this embodiment, first, for each word, it is represented as a query and a key, and an aspect vector is introduced as an additional input to enhance the semantic information related to the specified aspect; then, based on the correlation and aspect information in the text, the attention weights between words are calculated, and the distribution of attention is adjusted accordingly; finally, the adjusted attention distribution can accurately capture the semantic features of the specified aspect in the text, and generate a semantic vector for each word accordingly.
[0081] In one embodiment, the composition tree and dependency tree information of the training text are fused according to the encoding vector sequence to obtain a syntactic feature matrix of the training text, which specifically includes:
[0082] A word-level dependency adjacency matrix DA is constructed according to the dependency tree of the target text and the encoding vector sequence; a clause-level component adjacency matrix CA is constructed according to the composition tree of the target text and the encoding vector sequence; and a syntactic feature matrix EM is determined according to the dependency adjacency matrix DA and the component adjacency matrix CA.
[0083] Specifically, in this embodiment, there are two ways to determine the syntactic feature matrix EM: (1) If for each layer of the constituent tree Con.Tree, only the neighbors of the dependency tree Dep.Tree that are also in the same phrase are considered, the syntactic feature matrix EM = CA·DA. (2) If for each layer of the constituent tree Con.Tree, the words in the same phrase and the neighbors of the dependency tree Dep.Tree are considered, some edges of the dependency tree Dep.Tree can shorten the path between the aspect words and the related opinion words, and the syntactic feature matrix EM = CA+DA.
[0084] In one embodiment, the graph attention network is assumed to include L graph attention layers connected in series, and the number of attention heads is Z.
[0085] Will As the input of the first layer, the relationship between node i and its neighbor node j in the lth layer is calculated according to the following formula
[0086]
[0087] in, It represents the feature vector extracted from the row where a specific word is located in the EM matrix. It contains the input target text and specific information; It represents the set of neighbor nodes of node i in layer l when the time step is not considered. represents the final representation of node i in the l-1 layer, f is the scoring function that measures the relevance of two words, j and j' represent different neighbor nodes in the neighbor node set, and t represents the time step;
[0088] The masked attention mechanism is used to aggregate the information of all neighbor nodes of node i according to the following formula to obtain the intermediate representation of node i in layer l:
[0089]
[0090] in, represents the learnable parameters of the z-th head in the l-th layer, σ represents the activation function, and ∥ represents vector concatenation; It represents the set of neighbor nodes of node i in layer l when considering the time step;
[0091] The final representation of node i in the l-1 layer and the intermediate representation of node i in the l layer are mapped using a fully connected feedforward network according to the following formula to obtain the final representation of node i in the l layer;
[0092]
[0093] Among them, FC represents a fully connected feed-forward network.
[0094] Specifically, in this embodiment, the graph attention network GAT uses the attention mechanism to transfer and update information between nodes in the graph. This process starts from the bottom of the composition tree, proceeds upward layer by layer, and finally reaches the top of the composition tree. Through layer-by-layer aggregation, each word not only obtains local (i.e., within the phrase) information, but also obtains global (i.e., the entire sentence) syntactic information. At the same time, the dependency tree is used to capture the direct syntactic dependencies between words, which helps to understand the relationship between words. Finally, these two aspects of information are fused together to obtain the complete syntactic structure information of each aspect of the word. Figure 1 Taking the input text in as an example, the embodiment of the present invention can better align the emotion-representing word "excellent" in the target text with the aspect words it modifies through the syntactic tree structure.
[0095] In one embodiment, the post-interaction features of the training text include a post-interaction semantic vector and a post-interaction syntactic vector:
[0096] H syn′ =softmax(H sem W1(H syn ) T )H syn ;
[0097] H sem′ =softmax(H syn W2(H sem ) T )H sem ;
[0098] Among them, H syn′ and H syn Represent the syntactic vectors before and after the interaction, H sem′ and H sem They represent the semantic vectors after and before the interaction respectively, W1 and W2 represent learnable weight matrices, and softmax represents the normalized activation function.
[0099] It can be understood that the BiAffine attention mechanism converts the syntactic encoding H syn and semantic encoding H sem The information from these two different levels is fused in a complex space to generate a new representation that reflects richer semantic and syntactic structure features for subsequent sentiment analysis.
[0100] In one embodiment, the sentiment analysis module includes a maximum pooling layer, a multi-layer perceptron and a softmax layer; correspondingly, based on the post-interaction features input into the sentiment analysis module, the sentiment polarity of each aspect of the words in the training text is predicted, specifically including: the post-interaction features are first subjected to a maximum pooling operation by a maximum pooling layer, then mapped by a multi-layer perceptron, and finally subjected to sentiment classification by a softmax layer, and the sentiment polarity of each aspect of the words is predicted, and the prediction formula is as follows: P(a)=softmax(W p H α +b p )
[0101] Among them, P(a) is the probability distribution of predicted sentiment polarity, softmax() is the normalized activation function, and W p and b p Represents the learnable weight matrix and bias.
[0102] In one embodiment, the loss between the predicted sentiment polarity and the true sentiment label is calculated according to the following formula:
[0103]
[0104] Where D represents the training dataset, s represents the training text, a represents all aspect words in s, (s, a) represents the sentence aspect pair (s, a), C is the sentiment label set, θ represents all learnable parameters in the aspect-level sentiment analysis network, y(a) represents the true sentiment label corresponding to a, p(a) represents the predicted sentiment polarity, and loss represents the cross entropy loss.
[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An aspect-level sentiment analysis method based on composition tree and multiple attention mechanism, characterized in that: include: Step 1: construct an aspect-level sentiment analysis network, including a feature extraction module based on a multi-attention mechanism, a BiAffine module and a sentiment analysis module; the feature extraction module includes a self-attention mechanism module, an aspect-aware attention mechanism module and a graph attention network; Step 2: construct a training data set; each training sample in the training data set includes a training text, an aspect word in the training text, and a sentiment label corresponding to the aspect word; Step 3: Generate an encoding vector sequence of the training text, and generate a semantic vector of the training text using a self-attention mechanism module and an aspect-aware attention mechanism module according to the encoding vector sequence; wherein each vector in the encoding vector sequence corresponds to a word in the training text; Step 4: Generate a composition tree and a dependency tree of the training text, and fuse the information of the composition tree and the dependency tree of the training text according to the encoding vector sequence to obtain a syntactic feature matrix of the training text; Step 5: Take each word as a node, and use the graph attention network to aggregate neighbor node information according to the syntactic feature matrix of the training text to generate a syntactic vector of the training text; Step 6: Input the semantic vector and the syntactic vector of the training text into the BiAffine module to obtain the interactive features of the training text; Step 7: Based on the post-interaction features, the sentiment analysis module is input to predict the sentiment polarity of each aspect of the training text, and the loss between the predicted sentiment polarity and the real sentiment label is minimized to optimize the parameters of the aspect-level sentiment analysis network to obtain the aspect-level sentiment analysis model; Step 8: Input the target text and the aspect words in the target text into the aspect-level sentiment analysis model to obtain the sentiment polarity of the aspect words in the target text.
2. The aspect-level sentiment analysis method based on composition tree and multiple attention mechanism according to claim 1, characterized in that: In step 3, the BERT encoder is used to generate the encoding vector sequence of the training text.
3. The aspect-level sentiment analysis method based on composition tree and multiple attention mechanism according to claim 1, characterized in that: In step 3, according to the encoding vector sequence, a semantic vector of the training text is generated using a self-attention mechanism module and an aspect-aware attention mechanism module, specifically including: In the self-attention mechanism module, the self-attention weight matrix A of the training text is calculated according to the following formula: self ; Where Q and K represent the query matrix and key matrix composed of word element vectors, d represents the dimension of word element vector, and W Q and W K The learnable weight matrices corresponding to Q and K respectively; In the aspect-aware attention mechanism module, the aspect-aware attention weight matrix A of the training text is calculated according to the following formula: sap ; A asp =tanh(H a W a ×(KW k ) T +b); Among them, H a represents the aspect representation determined based on all aspect word vectors in the training text; K represents the encoding vector sequence of the training text, W a Indicates and W k Respectively represent H a The learnable weight matrix corresponding to K, b represents the learnable bias; tanh represents the activation function; Generate the semantic vector H of the training text according to the following formula sem ; H sem =A self +A asp 。 4. The aspect-level sentiment analysis method based on composition tree and multiple attention mechanism according to claim 1, characterized in that: In step 4, the composition tree and dependency tree information of the training text are fused according to the encoding vector sequence to obtain the syntactic feature matrix of the training text, which specifically includes: Constructing a word-level dependency adjacency matrix DA according to the dependency tree of the target text and the encoding vector sequence; Constructing a clause-level component adjacency matrix CA according to the component tree of the target text and the encoding vector sequence; A syntactic feature matrix is determined according to the dependency adjacency matrix DA and the component adjacency matrix CA.
5. The aspect-level sentiment analysis method based on composition tree and multiple attention mechanism according to claim 1, characterized in that: Step 5 specifically includes: Assume that the graph attention network consists of L graph attention layers connected in series, with the number of attention heads being Z. Will As the input of the first layer, the relationship between node i and its neighbor node j in the lth layer is calculated according to the following formula in, It represents the feature vector extracted from the row of a specific word in the syntactic feature matrix. It represents the set of neighbor nodes of node i in layer l when the time step is not considered. represents the final representation of node i in the l-1 layer, f is the scoring function that measures the relevance of two words, j and j' represent different neighbor nodes in the neighbor node set, and t represents the time step; The masked attention mechanism is used to aggregate the information of all neighbor nodes of node i according to the following formula to obtain the intermediate representation of node i in layer l: in, represents the learnable parameters of the zth head in the lth layer, σ represents the activation function, ∥ represents vector concatenation, It represents the set of neighbor nodes of node i in layer l when considering the time step; The final representation of node i in the l-1 layer and the intermediate representation of node i in the l layer are mapped using a fully connected feedforward network according to the following formula to obtain the final representation of node i in the l layer; Among them, FC represents a fully connected feed-forward network.
6. The aspect-level sentiment analysis method based on composition tree and multiple attention mechanism according to claim 1, characterized in that: In step 6, the post-interaction features of the training text include the post-interaction semantic vector and the post-interaction syntactic vector: H syn′ =softmax(H sem W1(H syn ) T )H syn ; H sem′ =softmax(H syn W2(H sem ) T )H sem ; Among them, H syn′ and H syn Represent the syntactic vectors before and after the interaction, H sem′ and H sem They represent the semantic vectors after and before the interaction respectively, W1 and W2 represent learnable weight matrices, and softmax represents the normalized activation function.
7. The aspect-level sentiment analysis method based on composition tree and multiple attention mechanism according to claim 1, characterized in that: In step 7, the sentiment analysis module includes a maximum pooling layer, a multi-layer perceptron and a softmax layer; correspondingly, based on the post-interaction features input to the sentiment analysis module, the sentiment polarity of various aspects of the words in the training text is predicted, specifically including: The interactive features are first subjected to a maximum pooling operation by a maximum pooling layer, then mapped by a multi-layer perceptron, and finally subjected to sentiment classification by a softmax layer to predict the sentiment polarity of each aspect of the words.
8. The aspect-level sentiment analysis method based on composition tree and multiple attention mechanism according to claim 1, characterized in that: In step 7, the loss between the predicted sentiment polarity and the true sentiment label is calculated according to the following formula; Where D represents the training dataset, s represents the training text, a represents all aspect words in s, (s, a) represents the sentence aspect pair (s, a), C is the sentiment label set, θ represents all learnable parameters in the aspect-level sentiment analysis network, y(a) represents the true sentiment label corresponding to a, p(a) represents the predicted sentiment polarity, and loss represents the cross entropy loss.
Citation Information
Cited By
Sentiment analysis method and system based on conversation attention mechanism and double-feature fusion
CN120067335A
Emotional Analysis Method and System Based on Conversation Attention Mechanism and Dual-Feature Fusion
CN120067335B
