Emotion analysis method of heterogeneous graph convolutional network based on dependency enhancement

Abstract

The invention discloses an emotion analysis method based on a dependency-enhanced heterogeneous graph convolutional network. The method comprises the following steps: preprocessing a to-be-analyzed text; performing feature extraction on the word vector embedding representation of the context and aspect of the analysis text by using a bidirectional long-short memory network Bi-LSTM, and inputting a word embedding matrix into the Bi-LSTM to obtain a hidden layer state vector of a sentence; the dependency syntactic graph convolutional network further extracts dependency syntactic features in the sentences, and the component syntactic graph convolutional network further extracts component syntactic features; after the input sentence is processed by the two GCNs, the outputs of the two different graph convolutional networks are connected to obtain more comprehensive feature representation; in the interactive attention layer, the model learns the correlation between words and other words, so that the relationship and mode in the input data are more comprehensively understood; and the interactive attention layer is subjected to classification output through an input softmax normalization layer. According to the method, the accuracy of sentence extreme value judgment can be improved.

Description

Technical Field

[0001] The present invention relates to sentiment analysis technology, in particular to a sentiment analysis method based on a dependency-enhanced heterogeneous graph convolutional network. Background Art

[0002] With the rapid development of science and technology, users have posted many comments expressing their opinions online. By mining the emotional expressions in the comments, we can analyze the main points of interest of users. Text sentiment analysis can be divided into chapter-level sentiment analysis, sentence-level sentiment analysis, and aspect-level sentiment analysis according to the analysis granularity. At present, the relevant research technology for text sentiment analysis of the entire text can achieve excellent performance, but this overall sentiment analysis can only express the overall sentiment trend of the text, which belongs to coarse-grained text sentiment analysis. In practical applications, in order to analyze the specific preferences of users, it is necessary to analyze the sentiment polarity of specific aspects for sentences. Therefore, as a fine-grained sentiment analysis task, aspect-based sentiment analysis (ABSA) has become a popular research direction in natural language processing.

[0003] Aspect-level sentiment analysis refers to determining the sentiment polarity (neutral, positive, negative) of a specific aspect of a sentence. For example, "The fruit in this store is very fresh, but the price is too expensive", where "fruit" belongs to the positive sentiment category and "price" belongs to the negative sentiment category. Methods for aspect-level sentiment analysis mainly include methods based on sentiment dictionaries, methods based on machine learning, and methods based on deep learning. The dictionary-based method mainly evaluates the sentiment polarity in the text through a predefined sentiment vocabulary. Since the speed of collecting sentiment vocabulary cannot keep up with the speed of the emergence of new words on the Internet, the dictionary-based method often affects the accuracy of the analysis results.

[0004] Traditional machine learning methods learn by automatically identifying rules in sentences. They mainly include linear models, support vector machines, and decision trees to classify and predict data. These methods perform well when processing structured data, but are often limited when faced with complex language features and contextual information. In the face of these problems, deep learning methods rooted in neural networks provide a promising approach to handling complex sentiment analysis tasks. Deep learning methods are often used in sentiment analysis because of their ability to handle complex structures and automatically learn features. The neural networks used in deep learning include convolutional neural networks, recurrent neural networks, and graph neural networks.

[0005] However, existing GCN sentiment analysis methods focus more on the sentiment polarity of a single word, while ignoring the syntactic structure and dependency relationship of the context. Existing research shows that in sentiment analysis, syntactic structure and dependency relationship are very important for the model to judge the polarity of sentences. Therefore, integrating syntactic structure and dependency relationship into the graph network can enhance the model's understanding of text semantics, improve the accuracy of sentiment analysis and the ability to capture fine-grained features. Summary of the invention

[0006] The purpose of the present invention is to provide a sentiment analysis method based on a heterogeneous graph convolutional network with dependency enhancement to address the deficiencies of the prior art. This method can improve the accuracy of sentence extreme value judgment.

[0007] The technical solution for achieving the purpose of the present invention is:

[0008] A sentiment analysis method based on a heterogeneous graph convolutional network with dependency enhancement includes the following steps:

[0009] 1) Preprocessing: The text to be analyzed is preprocessed and converted into word vector embedding form using the global vector word representation (GloVe) word vector model. Specifically:

[0010] According to a 300-dimensional pre-trained word vector GloVe, the input sentence Convert to word embedding matrix:

[0011]

[0012] where d e represents the dimension of word embedding, |n| is the number of words in the sentence, and then the word embedding matrix V is input into the bidirectional LSTM, i.e., Bi-LSTM, to generate the hidden layer state vector of the sentence:

[0013] H = {h1, h2, ..., h τ+1 , ..., h τ+m , ..., h n},

[0014] here, represents the hidden state vector of the i-th word, d h Represents the output dimension of unidirectional LSTM;

[0015] 2) Obtaining the hidden state vector of the sentence: Using a bidirectional long short-term memory network Bi-LSTM to extract features from the context information of the analysis text and the word vector embedding representation of the aspect words, the word embedding matrix is ​​input into the Bi-LSTM to obtain the hidden state vector of the sentence;

[0016] 3) Extract dependency syntactic features and component syntactic features: Use dependency syntactic graph convolutional network to further extract dependency syntactic features in sentences, and use component graph convolutional network to further extract component syntactic features, including:

[0017] 1-3) The Dependency Syntax Graph Convolutional Network (DepGCN) module is used to extract dependency syntactic features. DepGCN takes syntactic encoding as input and uses the dependency arc probability matrix generated by the dependency parser to encode syntactic information. The dependency graph uses the adjacency matrix A dep ∈R n×n It is defined as:

[0018]

[0019] Among them, link(ij) indicates that there is a dependency relationship between the i-th word and the j-th word;

[0020] 2-3) According to the formula:

[0021]

[0022] Generate dependency graph representation through DepGCN module:

[0023]

[0024] Among them, W l represents the weight matrix, b l represents the bias term, σ is the activation function, is the hidden state vector of the i-th word at layer l;

[0025] 3-3) The component syntactic graph convolutional network (SynGCN) module is used to extract component syntactic features and construct the grammatical structure of the component composition tree in a bottom-up manner. The component composition tree consists of multiple phrases of the input text. Composition, based on these phrases Create the corresponding graph. Considering the depth of the composition tree, a total of m layers are selected and the interval is set to 3. This is because the change of phrase level information between adjacent layers is small, and over-alignment will lead to inefficient use of computing resources. In addition, the value of m is consistent with the number of SynGCN layers. The composition graph is constructed through the adjacency matrix A syn ∈R lc×n×n It means that the definition is as follows:

[0026]

[0027] Where m represents the selected lth c The level of the phrase within the layer, and u represents the component tags associated with the phrase, namely S, NP, VP;

[0028] The formula adopted in 4-3):

[0029]

[0030] Generate the output hidden representation through the SynGCN module:

[0031]

[0032] Here, W l represents the weight matrix, b l represents the bias term, σ is the activation function, is the hidden state vector of the i-th word at the l-th layer;

[0033] 4) Joint features: Connect the outputs of the dependency syntactic graph convolutional network and the constituent syntactic graph convolutional network to obtain a more comprehensive feature representation. At the same time, mask the hidden state vectors of non-aspect words and keep the state of aspect words unchanged to ensure that only the part related to the aspect is retained, as follows:

[0034] 1-4) After the input sentence is processed by two different GCN modules, generate a mask matrix through the mask layer to screen the output. Subsequently, connect the outputs of the two graph convolutional networks, which is expressed as follows:

[0035] h ds = [h d ; h s , where h d and h s respectively represent the output features of the two graph convolutional networks, and h ds is the final joint feature representation;

[0036] 2-4) Mask the hidden state vectors of non-aspect words and keep the state of aspect words unchanged, including:

[0037] To highlight the feature information of aspect words in the sentence, this technical solution adopts masking the hidden state vectors of non-aspect words and only retaining the state of aspect words, so as to ensure that only the part related to the aspect is retained. The specific implementation is as follows:

[0038] When the condition 1 ≤ t < τ + 1 or τ + m < t ≤ n is satisfied, the hidden state vectors of non-aspect words are set to 0, that is:

[0039]

[0040]

[0041]

[0042] Among them, to The hidden state vector of the aspect word remains unchanged, while the hidden state vectors of other positions are masked;

[0043] 5) Normalization: In the interactive attention layer of the attention mechanism network, the model learns the correlation between words and other words, and more comprehensively understands the semantic structure and sentiment information in the input data. The interactive attention layer is input into the softmax normalization layer to obtain the classification output and obtain the final feature prediction sentiment polarity, including:

[0044] 1-5) Set attention weights for each context word in the sentence, including:

[0045] Based on the aspect-oriented features, an attention mechanism based on self-interaction attention is adopted to generate a refined representation of the hidden state vector H. The core idea is to learn important features related to the semantics of aspect words from the hidden state vector and assign attention weights based on self-interaction attention to each context word. The calculation process of the attention weight is:

[0046] First, the weight β is calculated by dot product t , to measure the semantic relevance between the aspect word and each word in the sentence:

[0047]

[0048] Next, β t Normalize to get the attention weight α of each word t :

[0049]

[0050] Among them, the dot product is used to capture the semantic correlation between aspect words and context words, thereby achieving feature masking for specific aspects, such as zero masking;

[0051] Finally, combined with the attention weight α t , calculate the overall representation r of the prediction:

[0052]

[0053] 2-5) Input the output r of the interactive attention layer into the fully connected layer to generate the classification result. The formula for predicting the sentiment polarity is:

[0054] p=softmax(W p r+b p ),

[0055] Among them, W p is the weight matrix, b p is the bias term, and p is the sentiment category distribution obtained by softmax normalization;

[0056] The loss function of classification consists of cross entropy and L2 regularization terms. The specific formula is as follows:

[0057]

[0058] Among them, C represents the number of categories of sentence classification, y i Indicates the true sentiment category of the sentence, p i Represents the predicted sentence sentiment category, λ represents the weight of the L2 regularization term, and θ represents all trainable parameters. This design can effectively optimize the model performance while suppressing overfitting through regularization.

[0059] This mechanism effectively highlights the contextual features related to aspects and provides more accurate representation for the final task.

[0060] After converting the data into word vectors through Glove, this technical solution constructs a syntactic structure dependency graph and a dependency dependency graph based on the syntactic structure and dependency relationship, and integrates them through the self-interactive attention mechanism to obtain rich contextual information in the deep learning hybrid model built based on GCN, thereby improving the accuracy of sentence extreme value judgment.

[0061] Compared with the prior art, the beneficial effects of this technical solution are:

[0062] By adopting a bidirectional graph convolutional network based on dependency and component structure, the syntactic structure graph and the dependency graph can be constructed respectively. This technical solution adopts different perspectives of dependency tree and component tree to deeply capture the grammatical structure information of the sentence. The dependency graph focuses on directly describing the grammatical dependencies between words, while the component structure graph reveals the relationship between phrases and hierarchical structures. In the operation of the graph convolutional network (GCN), features are extracted from the two graph structures respectively, and node information is aggregated layer by layer to extract deeper grammatical features. Subsequently, the outputs of the two graph convolutional networks are connected together, and the feature representations of the two grammatical graphs are fused to construct a more comprehensive and richer feature vector representation. This fusion strategy can make full use of the complementary information of syntactic structure and dependency, and provide more accurate input features for subsequent tasks.

[0063] This method can improve the accuracy of sentence extreme value judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 is a flow chart of an embodiment;

[0065] Figure 2 is a syntactic dependency diagram in an embodiment;

[0066] Figure 3 Schematic diagram of the adjacency matrix in the embodiment. DETAILED DESCRIPTION

[0067] The content of the present invention is further described below in conjunction with the drawings and embodiments, but the present invention is not limited thereto.

[0068] Example:

[0069] Reference Figure 1 , a sentiment analysis method based on dependency-enhanced heterogeneous graph convolutional network, comprising the following steps:

[0070] 1) Preprocessing: Preprocess the text to be analyzed and use the global vector word representation GloVe word vector model to convert the text to be analyzed into a word vector embedding form. Specifically:

[0071] According to a 300-dimensional pre-trained word vector GloVe, the input sentence Convert to word embedding matrix:

[0072]

[0073] where d e represents the dimension of word embedding, |n| is the number of words in the sentence, and then the word embedding matrix V is input into the bidirectional LSTM, i.e., Bi-LSTM, to generate the hidden layer state vector of the sentence:

[0074] H = {h1, h2, ..., h τ+1 , ..., h τ+m , ..., h n},

[0075] here, represents the hidden state vector of the i-th word, d h Represents the output dimension of unidirectional LSTM;

[0076] In this example, the specific process is:

[0077] 1-1) Perform word segmentation on the analyzed text;

[0078] 2-1) Remove noise characters and keep only numbers, letters, hyphens and punctuation marks;

[0079] 3-1) Convert all letters to lowercase;

[0080] In this example, given a sentence S containing n words, S = {ω1,ω2…ω τ+1 …ω τ+m …ω n-1 ,ω n}, where {ω τ+1 ,…ω τ+m} represents the aspect word or aspect phrase in the sentence. On this basis, the input text is converted into word vector embedding. The 300-dimensional pre-trained GloVe word vector is used to map each word of the input sentence to the corresponding word vector space, thereby constructing a word embedding matrix: where d e Indicates the dimension of each word vector, which is 300 in this example. |n| indicates the number of words in the sentence, and |V| indicates the size of the vocabulary, that is, the number of all different words in the analyzed text. This method can effectively convert the vocabulary information in the text into a vector representation that can be processed by the model, making it easier for subsequent deep learning models to be trained and predicted.

[0081] 2) Obtain the hidden state vector of the sentence: Use the bidirectional long short-term memory network Bi-LSTM to extract the context information of the text and the word vector embedding representation of the aspect words, input the word embedding matrix into the Bi-LSTM, and obtain the hidden state vector of the sentence. In this example, the word embedding matrix V is input into the Bi-LSTM model. Bi-LSTM can capture the semantic dependency of the word sequence in the input text by considering the contextual information before and after (i.e., forward and backward LSTM processing). Through the joint training of forward and backward LSTM, Bi-LSTM can extract the contextual feature representation of each word in the sentence. In the preprocessing process of the model, each word ω i will be mapped to a d e dimensional word vectors, which will be used as the input sequence of Bi-LSTM. After the forward propagation and back propagation of Bi-LSTM, the hidden layer state vector of each word in the sentence is obtained, which is recorded as:

[0082] H = {h1, h2, ..., h τ+1 , ..., h τ+m , ..., h n},

[0083] Among them, h i The corresponding word is ω i The hidden layer state vector, h i It is a representation calculated by the Bi-LSTM network based on its context information. This hidden layer state vector not only retains the semantic information of the word itself, but also integrates the dependency between the word and other words in its context, providing rich feature representation for subsequent sentiment analysis or other natural language processing tasks;

[0084] 3) Extract dependency syntactic features and component syntactic features: Use the dependency syntactic graph convolutional network to further extract the dependency syntactic features in the sentence, and use the component syntactic graph convolutional network to further extract the component syntactic features.

[0085] SuPar dependency parser is used to parse each sentence and generate the corresponding dependency tree. Based on the parsing results, such as Figure 2 Specifically, if two words have a dependency relationship in the SuPar parsing result, an edge is established between them and the edge weight is 1; if there is no direct dependency relationship between the two words, the edge weight is 0, as shown in Figure 3 As shown;

[0086] In order to further optimize the structural expression of the dependency tree and enhance the semantic differentiation ability of edge weights, part of speech (POS tags) are introduced as the basis for adjustment. Different weights are assigned to different part of speech categories according to the part of speech standards, namely nouns, verbs, and adjectives, so as to refine the distribution of edge weights. Dependencies between important parts of speech can be assigned higher weights, while dependencies on minor modifiers may be assigned lower weights. This part of speech-based weighting strategy can more accurately reflect the key grammatical and semantic relationships in sentences.

[0087] The resulting adjacency matrix A not only truly reflects the dependency structure of the sentence, but also further enriches the detailed expression of the dependency tree by combining the part-of-speech weights, providing more accurate and diverse input features for subsequent feature extraction and deep model learning; specifically:

[0088] 1-3) Use the dependency syntax graph convolutional network to construct a granular dependency syntax feature module; use the dependency syntax graph convolutional network DepGCN module to extract syntactic features. DepGCN takes the syntax encoding as input and uses the dependency arc probability matrix generated by the dependency parser to encode the syntactic information. The dependency graph uses the adjacency matrix A dep ∈R n×n It is defined as:

[0089]

[0090] Among them, link(i, j) indicates that there is a dependency relationship between the i-th word and the j-th word;

[0091] 2-3) According to the formula:

[0092]

[0093] Generate dependency graph representation through DepGCN module:

[0094]

[0095] Among them, W l represents the weight matrix, b l represents the bias term, σ is the activation function, is the hidden state vector of the i-th word at layer l;

[0096] 3-3) Use the component syntax graph convolutional network to build a granular component syntax feature module; construct the grammatical structure of the component composition tree in a bottom-up manner, which consists of multiple phrases of the input text Composition, based on these phrases Create the corresponding graph. Considering the depth of the composition tree, a total of m layers are selected and the interval is set to 3. This is because the change of phrase level information between adjacent layers is small, and over-alignment will lead to inefficient use of computing resources. In addition, the value of m is consistent with the number of SynGCN layers. The composition graph is constructed through the adjacency matrix A con ∈R lc×n×n It is defined as follows:

[0097]

[0098] Where m represents the selected lth c The level of the phrase within the layer, and u represents the component tags associated with the phrase, namely S, NP, VP;

[0099] 4-3) The formula used is:

[0100]

[0101] Generate output hidden representation through SynGCN module:

[0102]

[0103] Here, W l represents the weight matrix, b l represents the bias term, σ is the activation function, is the hidden state vector of the i-th word at layer l;

[0104] 4) Joint features: Connect the outputs of the dependency syntax graph convolutional network and the component syntax graph convolutional network to obtain a more comprehensive feature representation. At the same time, shield the hidden state vectors of non-aspect words and keep the state of aspect words unchanged, ensuring that only the part related to the aspect is retained. Specifically:

[0105] 1-4) After the input sentence is processed by two different GCN modules, a mask matrix is ​​generated through the mask layer to filter the output. Subsequently, the outputs of the two graph convolutional networks are connected, as shown below:

[0106] h ds =[h d ;h s ],

[0107] Among them, h d and hs respectively represent the output features of two graph convolutional networks, h ds is the final joint feature representation;

[0108] 2-4) Mask the hidden state vectors of non-aspect words and keep the aspect word states unchanged, including:

[0109] To highlight the feature information of aspect words in the sentence, this technical solution uses masking the hidden state vectors of non-aspect words and only retaining the states of aspect words, so as to ensure that only the aspect-related parts are retained. The specific implementation is as follows:

[0110] When the condition 1≤t<τ+1 or τ+m<t≤n is satisfied, the hidden state vectors of non-aspect words are set to 0, that is:

[0111]

[0112] On this basis, the output of the vertical and horizontal mask layer is expressed as:

[0113]

[0114] Among them, to are the hidden state vectors of aspect words, and the states remain unchanged, while the hidden state vectors in other positions are masked;

[0115] 5) Normalization: In the interactive attention layer of the attention mechanism network, the model learns the correlation between aspect words and other words, more comprehensively understands the semantic structure and sentiment information in the input data, and obtains the classification output by passing the interactive attention layer through the input softmax normalization layer, and obtains the final feature prediction sentiment polarity, including:

[0116] 1-5) Set attention weights for each context word in the sentence, including:

[0117] Based on the aspect-oriented features, an attention mechanism based on self-interactive attention is adopted to generate a refined representation of the hidden state vector H. The core idea is to learn important features related to the semantics of aspect words from the hidden state vector and assign attention weights based on self-interactive attention to each context word. The calculation process of the attention weights is as follows:

[0118] First, calculate the weight β t through the dot product to measure the semantic correlation between the aspect word and each word in the sentence:

[0119]

[0120] Then, normalize β t to obtain the attention weight α t of each word:

[0121]

[0122] Among them, the dot product is used to capture the semantic correlation between aspect words and context words, thereby achieving feature masking for specific aspects, such as zero masking;

[0123] Finally, combined with the attention weight α t , calculate the overall representation r of the prediction:

[0124]

[0125] 2-5) Input the output r of the interactive attention layer into the fully connected layer to generate the classification result and predict the sentiment polarity. The formula is:

[0126] p=softmax(W p r+b p ),

[0127] Among them, W p is the weight matrix, b p is the bias term, and p is the sentiment category distribution obtained by softmax normalization;

[0128] The loss function of classification consists of cross entropy and L2 regularization terms. The specific formula is as follows:

[0129]

[0130] Among them, C represents the number of categories of sentence classification, y i Indicates the true sentiment category of the sentence, p i Represents the predicted sentence sentiment category, λ represents the weight of the L2 regularization term, and θ represents all trainable parameters. This design can effectively optimize the model performance while suppressing overfitting through regularization.

[0131] This mechanism effectively highlights the contextual features related to aspects and provides more accurate representation for the final task.

Claims

1. A sentiment analysis method based on dependency-enhanced heterogeneous graph convolutional networks, characterized in that: It includes the following steps: 1) Preprocessing: Preprocess the text to be analyzed, and use the Global Vectors for Word Representation (GloVe) word vector model to convert the text to be analyzed into a word vector embedding form. Specifically: Convert the input sentence □ into a word embedding matrix according to a pre-trained 300-dimensional GloVe word vector: where d e represents the dimension of word embedding, |n| is the number of words in the sentence, and then the word embedding matrix V is input into the bidirectional LSTM, i.e., Bi-LSTM, to generate the hidden layer state vector of the sentence: H={h1,h2,...,h τ+1， ...,h τ+m ,...,h n }, here, represents the hidden state vector of the i-th word, d h Represents the output dimension of unidirectional LSTM; 2) Obtain the hidden layer state vector of the sentence: Use a Bidirectional Long Short-Term Memory (Bi-LSTM) network to extract features from the context information of the analyzed text and the word vector embedding representation of the aspect word. Input the word embedding matrix into the Bi-LSTM to obtain the hidden layer state vector of the sentence; 3) Extract dependency syntactic feature and constituent syntactic features: Use a dependency syntactic graph convolutional network to extract the dependency syntactic features in the sentence, and use a constituent syntactic graph convolutional network to extract the constituent syntactic features, including: 1-3) The dependency syntactic graph convolutional network DepGCN module is used to extract dependency syntactic features. DepGCN takes syntactic encoding as input and uses the dependency arc probability matrix generated by the dependency parser to encode syntactic information. The dependency graph uses the adjacency matrix A dep ∈R n×n It is defined as: Among them, link(i, j) indicates that there is a dependency relationship between the i-th word and the j-th word; 2-3) According to the formula: Generate a dependency graph representation through the DepGCN module: Among them, W l represents the weight matrix, b l represents the bias term, σ is the activation function, is the hidden state vector of the i-th word in the first layer; 3-3) The component syntactic graph convolutional network (SynGCN) module is used to extract component syntactic features and construct a grammatical structure of a component composition tree in a bottom-up manner. The component composition tree consists of multiple phrases of the input text. Composition, based on these phrases Create the corresponding graph, the depth of the component tree, select a total of m layers, and set the interval to 3. In addition, the value of m is consistent with the number of SynGCN layers. The component graph is constructed through the adjacency matrix A syn ∈R lc×n×n It is defined as follows: Where m represents the selected lth c The level of the phrase within the layer, and u represents the component tags associated with the phrase, namely S, NP, VP; 4-3) The formula used: Generate an output hidden representation through the SynGCN module: Here, W l represents the weight matrix, b l represents the bias term, σ is the activation function, is the hidden state vector of the i-th word in the first layer; 4) Combine features: Connect the outputs of the dependency syntactic graph convolutional network and the constituent syntactic graph convolutional network to obtain a more comprehensive feature representation. At the same time, mask the hidden state vectors of non-aspect words and keep the aspect word states unchanged to ensure that only the part related to the aspect is retained. Specifically as follows: 1-4) After the input sentence is processed by two different GCN modules, generate a mask matrix through a mask layer to screen the output. Subsequently, connect the outputs of the two graph convolutional networks, as shown below: h ds =[h d ;h s ], Among them, h d and h s They represent the output features of the two graph convolutional networks, h ds is the final joint feature representation; 2-4) Mask the hidden state vectors of non-aspect words and keep the aspect word states unchanged, including: When the condition 1 ≤ t < τ + 1 or τ + m < t ≤ n is satisfied, the hidden state vectors of non-aspect words are set to 0, that is: On this basis, the output of the vertical and horizontal mask layers is expressed as: in, to The hidden state vector of the aspect word remains unchanged, while the hidden state vectors of other positions are masked; 5) Normalization: In the interactive attention layer of the attention mechanism network, the model learns the correlation between the aspect word and other words, and more comprehensively understands the semantic structure and sentiment information in the input data. Obtain the classification output through the input softmax normalization layer in the interactive attention layer, and obtain the final feature prediction of the sentiment polarity, including: 1-5) Set attention weights for each context word in the sentence, including: Based on the aspect-oriented features, use an attention mechanism based on self-interactive attention to generate a refined representation of the hidden state vector H, learn important features related to the aspect word semantics from the hidden state vector, and assign attention weights based on self-interactive attention to each context word. The calculation process of the attention weights is as follows: First, the weight β is calculated by dot product t , to measure the semantic relevance between the aspect word and each word in the sentence: Next, β t Normalize to get the attention weight α of each word t : Among them, the dot product is used to capture the semantic correlation between aspect words and context words to achieve feature masking for specific aspects; finally, combined with the attention weight α t , calculate the overall representation r of the prediction: 2-5) Input the output r of the interactive attention layer into the fully connected layer to generate the classification result and predict the sentiment polarity. The formula is: p=softmax(W p r+b p ), Among them, W p is the weight matrix, b p is the bias term, and p is the sentiment category distribution obtained by softmax normalization; The loss function of the classification consists of cross-entropy and L2 regularization terms. The specific formula is as follows: Among them, C represents the number of categories of sentence classification, y i Indicates the true sentiment category of the sentence, p i represents the predicted sentence sentiment category, λ represents the weight of the L2 regularization term, and θ represents all trainable parameters.

Images