An aspect-level sentiment analysis method and device based on a graph convolution network and a medium

Abstract

The application provides an aspect-level sentiment analysis method and device based on a graph convolution network and a medium, wherein a text to be processed is input into a trained aspect-level sentiment analysis model for analysis to obtain a sentiment polarity classification result corresponding to the text to be processed; the aspect-level sentiment analysis model comprises an input module, an attention module, a graph convolution module for masking and aggregating features output by a previous stage, an adaptive weight module for weighting the aggregated features, and an output module; compared with the prior art, the graph convolution module is arranged in the aspect-level sentiment analysis model, the information of context nodes is aggregated after the extracted features are subjected to masking processing, and the key part contributing to sentiment analysis can be paid more attention to; the adaptive weight module is arranged, the output of the graph convolution module is subjected to attention weight distribution, the network is better endowed with adaptive capability, the output result is optimized, and the accuracy of sentiment classification is improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing and artificial intelligence technology, and in particular to an aspect-level sentiment analysis method, device and medium based on a graph convolutional network. Background Art

[0002] With the rise of social media and online platforms, the amount of content generated has grown exponentially, and this content contains a wealth of sentiment and opinion information. Sentiment analysis, also commonly known as opinion mining, is used to analyze factors such as human emotions, emotions, evaluations, and attitudes toward a target.

[0003] In the task of aspect-level sentiment analysis, research methods have evolved from traditional rule- and lexicon-based approaches, through machine learning, to deep learning techniques. Early research focused on determining sentiment polarity using sentiment lexicons or specific rule-based methods. For example, some researchers compared the performance of naive Bayes, maximum entropy models, and support vector machines in sentiment classification tasks, finding that support vector machines performed better. Other researchers have proposed a lexicon-based approach specifically for feature extraction of explicit and implicit sentiment polarity in text.

[0004] With the development of deep learning technology, aspect-level sentiment analysis tasks have begun to adopt more complex neural network models, such as recurrent neural networks and their variants, and convolutional neural networks. These models can capture the sequential characteristics and local contextual information of text, thereby improving the accuracy of sentiment analysis. For example, some researchers have proposed a recursive neural network based on dependency trees. This model can adaptively transmit sentiment based on the syntactic relationships and contextual information between words, effectively improving the performance of sentiment classification. Other researchers have proposed a composite model combining a bidirectional long short-term memory network and a support vector regression model for fine-grained sentiment analysis tasks.

[0005] In recent years, graph neural networks, particularly models such as graph convolutional networks and graph attention networks, have been widely used in aspect-level sentiment analysis due to their advantages in processing graph-structured data. These graph-based models effectively integrate syntactic dependency information and contextual features by constructing dependency graphs between words, further improving the performance of sentiment classification. For example, some researchers have proposed a model that combines an aspect-based attention mechanism with a graph convolutional neural network, focusing on the fusion of aspect-level features and contextual information. Other researchers have designed a graph convolutional neural network model that combines a special gating mechanism with a self-attention network, which can dynamically integrate information from word dependency graphs and latent graphs.

[0006] Although models based on traditional methods and deep learning have achieved success in specific scenarios, they have limitations in capturing deep semantic relationships and subtle sentiment differences in complex text structures, resulting in low accuracy of sentiment classification results. Summary of the Invention

[0007] The present invention provides an aspect-level sentiment analysis method, device and medium based on a graph convolutional network, the purpose of which is to improve the accuracy of sentiment classification.

[0008] To achieve the above objectives, the present invention provides an aspect-level sentiment analysis method based on a graph convolutional network, comprising:

[0009] Step 1: Get the text to be processed;

[0010] Step 2: Input the text to be processed into the trained aspect-level sentiment analysis model for analysis to obtain the sentiment polarity classification result corresponding to the text to be processed;

[0011] The aspect-level sentiment analysis model includes an input module for converting input text, an attention module for feature extraction, a graph convolution module for masking and aggregating features output from the previous level, an adaptive weight module for weighting the aggregated features, and an output module for outputting sentiment polarity classification results.

[0012] The input end of the input module is the input end of the aspect-level sentiment analysis model;

[0013] The output of the input module is connected to the input of the attention module;

[0014] The output of the attention module is connected to the input of the graph convolution module and the first input of the adaptive weight module;

[0015] The first output terminal of the graph convolution module is connected to the second input terminal of the adaptive weight module;

[0016] The second output terminal of the graph convolution module and the output terminal of the adaptive weight module are both connected to the input terminal of the output module;

[0017] The output end of the output module is the output end of the aspect-level sentiment analysis model.

[0018] Furthermore, before the text to be processed is input into the trained aspect-level sentiment analysis model for analysis, the following steps are also included:

[0019] Preprocess the text to be processed to obtain the preprocessed text;

[0020] The preprocessed text is input into the trained aspect-level sentiment analysis model for analysis.

[0021] Furthermore, the input module includes a conversion unit, an enhancement unit, and an encoding unit;

[0022] The preprocessed text is converted into multiple word embedding vectors through the conversion unit;

[0023] Through the enhancement unit, all word embedding vectors are combined with the introduced position embedding vector and part-of-speech embedding vector to obtain multiple enhanced vectors;

[0024] All enhanced vectors are encoded through the encoding unit to generate a hidden state matrix, which includes multiple word vectors.

[0025] Furthermore, by combining all word embedding vectors with the introduced position embedding vector and part-of-speech embedding vector through the enhancement unit, the expression of multiple enhanced vectors is obtained as follows:

[0026] E i =Concatenate(x i ,p i ,t i )

[0027] Among them, E i represents the i-th enhancement vector, x i represents the i-th word embedding vector, p i represents the embedding vector of the i-th position, t i represents the i-th part-of-speech embedding vector, and COncatenate(·) represents the connection function.

[0028] Further, the attention module includes aspect-aware attention unit, self-attention unit and merging unit;

[0029] In the aspect-aware attention unit, the word vector extracted from the hidden state matrix is ​​converted into a query vector through the learning matrix, and the query vector is calculated through the aspect-aware attention function to obtain the aspect-aware attention weight. The aspect-aware attention weight is used to weight the hidden state matrix to obtain the aspect-aware feature matrix.

[0030] In the self-attention unit, multiple learning matrices are combined with the hidden state matrix to obtain the query matrix, key matrix and value matrix. The query matrix, key matrix and value matrix are calculated through the activation function to obtain the self-attention weight matrix, and the self-attention weight matrix is ​​combined with the value matrix to obtain the self-attention perception feature matrix.

[0031] The merging unit is used to merge the aspect perception feature matrix and the self-attention perception feature matrix to obtain a comprehensive perception feature matrix.

[0032] Furthermore, the graph convolution module includes a mask construction unit and a graph convolution unit;

[0033] The mask construction unit is used to construct a mask matrix according to the syntactic dependency tree of the sentence, and the mask matrix is ​​used to perform mask processing on the comprehensive perception feature matrix;

[0034] The graph convolution unit performs graph convolution on the comprehensive perception feature matrix after mask processing to obtain multiple node feature matrices.

[0035] Furthermore, the adaptive weight module is used to weight the aggregated features through the aspect-level attention mechanism, including:

[0036] Calculate the similarity score between the comprehensive perceptual feature matrix and each word vector;

[0037] Use the similarity score to calculate the attention weight of each word vector;

[0038] The feature matrix of each node is weighted and summed by the attention weight to obtain multiple weighted node feature matrices.

[0039] Furthermore, the output module includes an aggregation layer, a linear transformation layer, and a nonlinear activation function layer;

[0040] The first input terminal of the aggregation layer is connected to the first output terminal of the graph convolution unit, and the second terminal of the aggregation layer is connected to the output terminal of the adaptive weight module;

[0041] The output end of the aggregation layer is connected to the input end of the linear transformation layer, the output end of the linear transformation layer is connected to the input end of the nonlinear activation function layer, and the output end of the nonlinear activation function layer is the output end of the aspect-level sentiment analysis model;

[0042] The aggregation layer is used to aggregate multiple node feature matrices and multiple weighted node feature matrices in series to obtain an aggregated feature matrix;

[0043] The linear transformation layer is used to perform linear transformation on the aggregated feature matrix to obtain the transformed features;

[0044] The nonlinear activation function layer is used to classify the transformed features into sentiment polarity and obtain the predicted probability distribution corresponding to each sentiment category in the text to be processed.

[0045] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, an aspect-level sentiment analysis method based on a graph convolutional network is implemented.

[0046] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements an aspect-level sentiment analysis method based on a graph convolutional network.

[0047] The above solution of the present invention has the following beneficial effects:

[0048] The present invention obtains a sentiment polarity classification result corresponding to the text to be processed by analyzing the trained aspect-level sentiment analysis model inputted by the acquired text to be processed; the aspect-level sentiment analysis model includes an input module for converting the input text, an attention module for feature extraction, a graph convolution module for masking and aggregating features outputted at the previous level, an adaptive weight module for weighting the aggregated features, and an output module for outputting the sentiment polarity classification result; compared with the prior art, the present invention can pay more attention to the key parts that contribute to sentiment analysis by setting a graph convolution module in the aspect-level sentiment analysis model to mask the extracted features and then aggregate the information of the context nodes; and setting an adaptive weight module to distribute the output of the graph convolution module with attention weights, so as to better give the network an adaptive ability, thereby optimizing the output result and further improving the accuracy of sentiment classification.

[0049] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of a flow chart of an embodiment of the present invention;

[0051] Figure 2 Schematic diagram of the structure of the aspect-level sentiment analysis model in an embodiment of the present invention;

[0052] Figure 3 This is the impact of different numbers of graph convolution units on the accuracy results of two data sets in an embodiment of the present invention;

[0053] Figure 4 This is the impact of different numbers of graph convolutional units on the macro-average F1 results on two datasets in an embodiment of the present invention;

[0054] Figure 5 A schematic diagram of the structure of a terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] To make the technical problems, technical solutions, and advantages to be solved by the present invention more clear, the following is a detailed description with reference to the accompanying drawings and specific embodiments. It is obvious that the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0056] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0057] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood broadly. For example, they may refer to a locking connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0058] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0059] In response to existing problems, the present invention provides an aspect-level sentiment analysis method, device and medium based on graph convolutional networks.

[0060] like Figure 1 As shown, an embodiment of the present invention provides an aspect-level sentiment analysis method based on a graph convolutional network, comprising:

[0061] Step 1: Obtain the text to be processed, that is, the original text to be used for sentiment analysis, such as the original conversation text of the review software;

[0062] Step 2: Input the text to be processed into the trained aspect-level sentiment analysis model for analysis to obtain the sentiment polarity classification result corresponding to the text to be processed. The sentiment polarity can be positive, negative, or neutral.

[0063] The aspect-level sentiment analysis model includes an input module for converting input text, an attention module for feature extraction, a graph convolution module for aggregating features output from the previous level, an adaptive weight module for weighting the aggregated features, and an output module for outputting sentiment polarity classification results.

[0064] The input end of the input module is the input end of the aspect-level sentiment analysis model;

[0065] The output of the input module is connected to the input of the attention module;

[0066] The output of the attention module is connected to the input of the graph convolution module and the first input of the adaptive weight module;

[0067] The first output terminal of the graph convolution module is connected to the second input terminal of the adaptive weight module;

[0068] The second output terminal of the graph convolution module and the output terminal of the adaptive weight module are both connected to the input terminal of the output module;

[0069] The output end of the output module is the output end of the aspect-level sentiment analysis model.

[0070] The specific process of model processing is as follows:

[0071] Input the original text, which is the original dialogue text of the review software, such as "The scenery of this place is so beautiful". After the embedding layer, a preliminary semantic vector representation is obtained; after the attention layer, the original attention matrix is ​​obtained; the original attention matrix is ​​processed by the syntactic mask matrix to obtain an enhanced semantic representation matrix, and then the node information aggregation method of the graph convolution layer is used to obtain the enhanced matrix; the enhanced matrix and the original attention matrix are weighted to obtain a weight matrix to integrate the original information; finally, the weight matrix and the enhanced matrix are dimensionally merged to obtain the final output matrix, and then after a series of activation function operations, the final sentiment analysis prediction probability distribution is obtained. The final sentiment analysis prediction distribution is positive sentiment polarity.

[0072] Specifically, before the text to be processed is input into the trained aspect-level sentiment analysis model for analysis, the following steps are also included:

[0073] Preprocess the text to be processed to obtain the preprocessed text;

[0074] The preprocessed text is input into the trained aspect-level sentiment analysis model for analysis.

[0075] In an embodiment of the present invention, preprocessing is used to clean the text to be processed into a format suitable for analysis, specifically including removing useless information (such as HTML tags), performing word form restoration, standardizing the text format (such as converting plural words into basic forms, converting all text to be processed into lowercase), etc.

[0076] The input module includes a conversion unit, an enhancement unit, and an encoding unit;

[0077] The preprocessed text is converted into multiple word embedding vectors through the conversion unit;

[0078] Through the enhancement unit, all word embedding vectors are combined with the introduced position embedding vector and part-of-speech embedding vector to obtain multiple enhanced vectors;

[0079] All enhanced vectors are encoded through the encoding unit to generate a hidden state matrix, which includes multiple word vectors.

[0080] In the embodiment of the present invention, the conversion unit is internally integrated with a pre-trained word embedding model such as Word2Vec or GloVe or FastText or BERT or ELMo, which is used to convert each word w in the pre-processed text into i Are mapped to a high-dimensional vector space to obtain word embedding vectors where d e Represents the dimension of word embedding vectors, which can capture the semantic information of words. Therefore, the word embedding vectors of the entire text can form an embedding matrix

[0081] In order to further enrich the representation of word vectors, the embodiment of the present invention introduces position embedding vectors and part-of-speech embedding vectors in the enhancement unit, and combines these embedding vectors with the embedding matrix Each word embedding vector in is combined to obtain multiple enhanced vectors, so that the enhanced vector of the i-th word embedding vector can be calculated by the following expression:

[0082] E i =Concatenate(x i ,p i ,t i )

[0083] Among them, E i represents the i-th enhancement vector, x i represents the i-th word embedding vector, p i represents the embedding vector of the i-th position, t i represents the i-th part-of-speech embedding vector, and Concatenate(·) represents the concatenation function.

[0084] In order to capture context information, the embodiment of the present invention applies a bidirectional long short-term memory network (Bi-LSTM) in the encoding unit to encode the enhanced vector and generate a series of context-sensitive word vectors. To form the hidden state matrix where d h represents the hidden layer dimension of the bidirectional long short-term memory network, n represents the number of words in the sentence, and represents the dimension of the hidden state.

[0085] Most preferably, the attention module includes an aspect-aware attention unit, a self-attention unit, and a merging unit;

[0086] In the aspect-aware attention unit, the word vector extracted from the hidden state matrix is ​​converted into a query vector through the learning matrix, and the query vector is calculated through the aspect-aware attention function to obtain the aspect-aware attention weight. The aspect-aware attention weight is used to weight the hidden state matrix to obtain the aspect-aware feature matrix.

[0087] In the self-attention unit, multiple learning matrices are combined with the hidden state matrix to obtain the query matrix, key matrix and value matrix. The query matrix, key matrix and value matrix are calculated through the activation function to obtain the self-attention weight matrix, and the self-attention weight matrix is ​​combined with the value matrix to obtain the self-attention perception feature matrix.

[0088] The merging unit is used to merge the aspect perception feature matrix and the self-attention perception feature matrix to obtain a comprehensive perception feature matrix.

[0089] Specifically, the aspect-aware attention mechanism focuses on the sentiment expression of specific aspect terms. Its design is based on the following mathematical formalization, and the aspect-aware attention unit is specifically used to:

[0090] Extract the word vector h associated with the aspect term from the hidden state matrix H by position index a ;

[0091] In order to capture the contextual information related to aspect terms, the embodiment of the present invention introduces a learnable learning matrix

[0092] By learning the matrix W a The extracted word vector h a Convert to query vector q a , the conversion expression is:

[0093] q a =h a W a

[0094] The query vector q is processed by the aspect-aware attention function a Calculate and get the aspect-aware attention weight A asp , aspect-aware attention weight is used to characterize the contribution of each word in a sentence to a specific aspect term, and the aspect-aware attention function is:

[0095] A asp =Softmax(q a K T )

[0096] Among them, K represents the key matrix, T represents the transposed matrix, and the Softmax activation function is used to normalize the attention weights;

[0097] Using the aspect-aware attention weight A asp The hidden state matrix H is weighted to obtain the aspect perception feature H for predicting the sentiment tendency of aspect terms. asp , the weighted expression is:

[0098] H asp =A asp H.

[0099] Specifically, the self-attention mechanism captures the complex relationships within a sentence by calculating the influence weight of each word on all other words in the sentence. This mechanism allows the model to adaptively focus on the key parts of the sentence to obtain richer semantic representations. The self-attention unit is specifically used to:

[0100] Use different learning matrices to combine with the hidden state matrix to generate the query matrix Q, key matrix K and value matrix V respectively:

[0101] The query matrix is: Q = HW Q ;

[0102] The bond matrix is: K = HW K ;

[0103] The value matrix is: V = HW V ;

[0104] Among them, W Q 、W K 、W V Both represent learnable learning matrices;

[0105] Based on the activation function, the self-attention weight matrix A is calculated by the query matrix Q, key matrix K and value matrix V as:

[0106]

[0107] Among them, d k Indicates the dimension of the key vector, which is used to scale the result of the dot product to prevent the gradient from disappearing due to excessive inner product values;

[0108] Combine the self-attention weight matrix with the value matrix to obtain the self-attention perception feature matrix H self , the expression is:

[0109] H self =AV

[0110] Among them, A represents the self-attention weight matrix and V represents the value matrix.

[0111] The embodiment of the present invention merges the aspect perception feature matrix and the self-attention perception feature matrix through a merging unit to obtain a comprehensive perception feature matrix H att , the expression is:

[0112] H att =H asp +H self

[0113] The model that integrates the aspect-aware attention feature matrix and the self-attention feature matrix can capture aspect-related information while making full use of the lexical relationships within the sentence.

[0114] In order to further enhance the model's understanding of the syntactic structure of sentences, an embodiment of the present invention introduces the use of syntactic masks in conjunction with graph convolution in the graph convolution module. This design utilizes the syntactic dependency information of the sentence and guides the graph convolution through the syntactic mask to capture and utilize the dependency relationship between words, thereby achieving effective processing of complex sentence structures.

[0115] Most preferably, the graph convolution module includes a mask construction unit and a graph convolution unit;

[0116] The mask construction unit is used to construct a mask matrix according to the syntactic dependency tree of the sentence, and the mask matrix is ​​used to perform mask processing on the comprehensive perception feature matrix;

[0117] The graph convolution unit performs graph convolution on the comprehensive perception feature matrix after mask processing to obtain multiple node feature matrices.

[0118] In the embodiment of the present invention, the syntactic mask is constructed based on the syntactic dependency tree of the sentence. For any two word pairs in the sentence, a syntactic mask matrix is ​​defined. The distance between the word pairs in the syntactic dependency tree represents the syntactic connection strength between the words. The construction process of the matrix is ​​as follows:

[0119] Construct a matrix and initialize it to an all-zero matrix;

[0120] For word pairs that have direct syntactic dependency in a sentence, the sum is set to 1, indicating that there is direct syntactic dependency between the two words;

[0121] For indirectly dependent word pairs, the mask value is adjusted according to their distance in the syntactic dependency tree. The shortest syntactic path length between each pair of words is calculated, and then the syntactic mask matrix M is constructed based on this distance. The calculation expression of the shortest syntactic path length is:

[0122]

[0123] Where k represents a predefined syntactic distance threshold, which is used to determine which syntactic connections between words are strong enough to be taken into consideration, and dij represents the binary weight assigned to each word pair, indicating whether they are syntactically dependent.

[0124] The graph convolution unit takes the hidden state matrix processed by the syntactic mask matrix as input and aggregates the input information through the graph convolution operation. The expression of the graph convolution operation is:

[0125]

[0126] in, represents the node feature matrix of the lth layer in the graph convolution unit, Represents the syntactic mask matrix M and the identity matrix I n The sum of is used to ensure that each node’s own features are also included. represents a diagonal matrix whose elements yes The sum of the i-th row in is used for normalization, W (l) represents the learning matrix of the lth layer, σ represents the nonlinear activation function;

[0127] Through the above calculations, each node not only obtains its own feature information, but also integrates the features of the nodes directly connected to it in the syntactic tree. This integrated information can improve the accuracy of sentiment analysis. The multi-layer graph convolution unit can learn the deep syntactic characteristics of the sentence and enrich the semantic representation of the sentence. After further processing by the graph convolution unit, multiple more accurate node feature matrices are obtained.

[0128] Specifically, the adaptive weight module is used to weight the aggregated features, including:

[0129] Calculate the similarity score between the comprehensive perceptual feature matrix and each word vector;

[0130] Use the similarity score to calculate the attention weight of each word vector;

[0131] The feature matrix of each node is weighted and summed by the attention weight to obtain multiple weighted node feature matrices.

[0132] Specifically, in the aspect-level attention mechanism, what is actually represented is the weight of each word's importance to a specific aspect. These weights are calculated by comparing the similarity between the representation of each word in the sentence and the aspect representation. Therefore, for each word vector, this embodiment of the present invention calculates the similarity score between it and the comprehensive perceptual feature matrix, which is expressed as:

[0133] S i =H att T H i

[0134] Among them, S i represents the similarity score;

[0135] The similarity score is used to calculate the attention weight of each word, ensuring that the sum of all weights is 1. In this way, the weight of each word represents its importance in attention. The calculation expression is:

[0136]

[0137] Among them, W asp Represents the attention weight, and N represents the total number of words:

[0138] After obtaining the attention weights, in order to further emphasize the words related to specific aspects, the node feature matrix output by the graph convolution unit is Weighted to form weighted output, the process can be expressed by the following formula:

[0139]

[0140] Among them, O weighted Represents the i-th weighted node feature matrix, W asp,i represents the attention weight of the i-th word.

[0141] Most preferably, the output module includes an aggregation layer, a linear transformation layer, and a nonlinear activation function layer;

[0142] The first input terminal of the aggregation layer is connected to the first output terminal of the graph convolution unit, and the second terminal of the aggregation layer is connected to the output terminal of the adaptive weight module;

[0143] The output end of the aggregation layer is connected to the input end of the linear transformation layer, the output end of the linear transformation layer is connected to the input end of the nonlinear activation function layer, and the output end of the nonlinear activation function layer is the output end of the aspect-level sentiment analysis model;

[0144] The aggregation layer is used to aggregate multiple node feature matrices and multiple weighted node feature matrices in series to obtain an aggregated feature matrix;

[0145] The linear transformation layer is used to perform linear transformation on the aggregated feature matrix to obtain the transformed features;

[0146] The nonlinear activation function layer is used to classify the transformed features into sentiment polarity and obtain the predicted probability distribution corresponding to each sentiment category in the text to be processed.

[0147] Specifically, in order to utilize these multi-level information, the node feature matrices output by all layers are aggregated with multiple weighted node feature matrices in a series manner in the aggregation layer. The aggregation expression is:

[0148]

[0149] Among them, O combined Represents the aggregated feature matrix, Concat(·) represents the concatenation operation on the feature dimension, Represents the node feature matrix of the Lth layer in the graph convolution unit;

[0150] Before finally mapping the integrated output to the emotion category, in order to reduce the risk of overfitting and enhance the generalization ability of the model, the aggregated feature matrix is ​​then linearly transformed through a linear transformation layer to obtain the transformed feature O output , the expression is:

[0151] O output =ReLU(dropout(W b ·O combined +b))

[0152] Among them, W b , b represent the weight and bias of the linear transformation layer, respectively. Dropout(·) is used to prevent overfitting, and ReLU(·) is a nonlinear activation function.

[0153] Then, a nonlinear activation function is applied to increase the expressive power of the model. The nonlinear activation function layer is used to classify the transformed features into sentiment polarity, and obtain the predicted probability distribution corresponding to each sentiment category in the text to be processed. The expression is:

[0154] P=softmax(W f O output +b f )

[0155] Among them, W f represents the weight of the nonlinear activation function softmax, b f Represents the bias term, and P represents the predicted probability distribution corresponding to each sentiment category in the text to be processed.

[0156] In this embodiment of the present invention, for each text, the loss function calculates the difference between the probability distribution of the model output and the distribution of the true label. Assuming that the output of the model for a certain text is a probability distribution, where the total number of categories is the probability that the model predicts that the current text belongs to the category, if the true label is (represented by one-hot encoding, that is, 1 in the correct category position and 0 in other positions), then the cross entropy loss can be expressed as:

[0157]

[0158] Among them, y iIs the value of the true label at the position of category i. If the current text actually belongs to category i, then y i =1, otherwise y i =0.

[0159] During training, the model parameters are adjusted by minimizing this loss function, so that the model's output P increasingly approximates the true label distribution y. In this way, the model learns how to correctly classify text. Furthermore, BERT-based models incorporate regularization terms to mitigate overfitting, such as using dropout in BERT's attention mechanism and hidden layers. These techniques help improve the model's generalization capabilities on unseen data, ultimately resulting in a trained aspect-level sentiment analysis model.

[0160] The present invention conducts experiments on aspect-based sentiment analysis on three benchmark datasets. The benchmark datasets include restaurant and laptop computer reviews from SemEval 2014 Task 4, as well as Twitter articles. Each aspect has one of three sentiment polarities: positive, neutral, and negative. The statistics of the three datasets are shown in Table 1 below:

[0161] Table 1

[0162]

[0163] To initialize the transformation unit, this embodiment uses pretrained GloVe vectors. These vectors are learned from a large text corpus and have a dimension of 300, providing the model with rich semantic information. Furthermore, the model incorporates a bidirectional long short-term memory network into the encoding unit after the transformation unit to capture contextual information in the sentence. The hidden state dimension in each direction is set to 50.

[0164] During training, this embodiment of the present invention uses the Adam optimizer and a learning rate of 0.002 to promote model convergence. The batch size is set to 16, allowing the model to process more data per training session while also maintaining a certain batch size to prevent overfitting. The dimension of the position embedding vector is set to 30 to help the model better understand the relative position of words in a sentence.

[0165] The model of the embodiment of the present invention includes a graph convolution unit composed of a graph convolutional network, which uses a dimension of 768 to encode the context information of each word. In order to further improve the performance of the model, a 5-head multi-head attention mechanism is adopted, and a dropout rate of 0.2 is set to avoid overfitting. To ensure the reproducibility of the experimental results, the random seed in the experiment is fixed at 1000, and the BERT pre-trained model is additionally used for parameter initialization, providing a powerful semantic representation foundation for the model.

[0166] To comprehensively evaluate the model of the present invention, the present invention selected the following classic baseline models for comparison:

[0167] RAM uses bidirectional LSTM and attention mechanism to learn sentence representation;

[0168] TNet uses a CNN model to extract salient features from target-specific embeddings through converted Bi-LSTM embeddings;

[0169] ASGCN constructs aspect attention and combines it with GCN to extract aspect features of sentences;

[0170] CDT uses convolution on dependency tree model to learn the representation of sentence features;

[0171] Bi-GCN proposes a global vocabulary graph to encode word co-occurrence information, and at the same time builds a concept hierarchy on the local syntactic graph and vocabulary graph to distinguish different types of co-occurrence relationships;

[0172] KumaGCN combines information from dependency graph and latent graph to learn grammatical features;

[0173] DGEDT proposes a graph-enhanced Transformer network, introducing dual affine transformations to interactively learn features from Transformer and graph-based information from GCN.

[0174] Baseline Model Results and Analysis

[0175] The model provided by the embodiment of the present invention and the baseline model were compared on three public datasets in terms of aspect-level sentiment analysis accuracy and macro-average F1. The results are shown in Table 2:

[0176] Table 2

[0177]

[0178]

[0179] As shown in Table 2, on the Laptop dataset, the SM-GCN model of the present invention leads with an accuracy of 79.43% and a macro-average F1 score of 76.38%, demonstrating a significant advantage over other baseline models. This excellent performance demonstrates the effectiveness of the SM-GCN model of the present invention in capturing specific sentiment information. On the Restaurant dataset, the model of the present invention also leads in accuracy (83.82%) and F1 score (77.02%). Compared with the Bi-GCN model, the GCN of the present invention achieves a nearly 2% improvement in accuracy, demonstrating its superior performance when processing user comments containing complex sentiment. On the Twitter dataset, despite the more informal and diverse textual characteristics of this dataset, the model of the present invention achieves a macro-average F1 score of 73.78%, demonstrating its robustness to more haphazardly structured text. In summary, the BERT-based model demonstrates superior performance across all datasets. In particular, the model of the present invention achieves significant improvement on the Twitter dataset. However, on the Restaurant dataset, the DGEDT+BERT model slightly outperforms with a macro-average F1 score of 80.00%, likely due to the unique applicability of the specific model architecture combined with BERT in this domain. Nevertheless, the model in this embodiment demonstrates the potential of deep semantic analysis combined with graph convolutional networks to improve results in sentiment classification tasks.

[0180] To gain a deeper understanding of the contribution of each component in the model, the present invention conducted a series of ablation experiments, removing key components of the model: the attention module, the syntactic mask matrix, and the adaptive weight module. The removed components are marked with "w / o". The specific experimental results are shown in Table 3:

[0181] Table 3

[0182]

[0183] First, when the attention module is removed, the model's accuracy on the Laptop dataset drops from 79.43% to 75.74%, and its F1 score drops from 76.38% to 70.98%. This significant performance drop indicates that the attention mechanism is crucial for capturing contextual information related to specific aspects. Performance on the Restaurant dataset also exhibits a similar downward trend, with accuracy dropping from 83.82% to 82.39% and F1 score dropping from 77.02% to 74.14%. Removing the syntactic mask matrix results in slight performance drops on the Laptop and Restaurant datasets, respectively, but a more significant drop on the Twitter dataset, with accuracy dropping from 75.33% to 75.04% and F1 score dropping slightly from 73.78% to 73.40%. This suggests that the impact of syntactic structure on the model varies depending on the dataset characteristics. For some datasets, the lack of syntactic information can negatively impact the model's understanding of text structure and semantics. Eliminating the adaptive weight module also has a significant impact on model performance. In particular, on the Twitter dataset, the model's accuracy dropped from 75.33% to 73.18%, and its F1 score dropped from 73.78% to 71.18%. This result reveals the important role that attention weights play in capturing sentiment and aspect-related details. In summary, ablation experiments show that each module contributes to the overall performance of the model.

[0184] Finally, we explore the impact of the number of GCN layers on the results

[0185] In order to explore the impact of different GCN layers on the results, the embodiment of the present invention sets the number of GCN layers to 1 to 4 in two data sets, and analyzes the impact of different layers on the accuracy and macro-average F1 results. Figure 3 , as shown in Figure 4. When exploring the impact of the number of GCN layers on the results, the embodiment of the present invention conducted a detailed analysis of the results of the two datasets, Restaurant and Twitter, which have relatively complex data structures. For the Restaurant dataset, experiments show that the configuration of a single-layer GCN can achieve optimal performance. As the number of GCN layers increases, it can be noticed that the performance of the model shows a downward trend. In particular, when the number of layers increases to four, the performance degradation is particularly obvious. This may be because the model begins to have the problem of over-smoothing, which makes the distinction between different nodes lower. On the Twitter dataset, the performance of the model under the single-layer GCN structure is also the best, and the increase in the number of layers will lead to a decline in performance, especially when the number of layers reaches four, the performance loss is significant. The embodiment of the present invention can be seen that in this type of task, too many layers may cause the model to be unable to effectively capture useful features, but instead damage the generalization ability of the model.

[0186] The embodiment of the present invention obtains the sentiment polarity classification result corresponding to the text to be processed by analyzing the acquired text to be processed input into the trained aspect-level sentiment analysis model; the aspect-level sentiment analysis model includes an input module for converting the input text, an attention module for feature extraction, a graph convolution module for masking and aggregating the features outputted at the previous level, an adaptive weight module for weighting the aggregated features, and an output module for outputting the sentiment polarity classification result; compared with the prior art, the embodiment of the present invention pays more attention to the key parts that contribute to sentiment analysis by setting a graph convolution module in the aspect-level sentiment analysis model to mask the extracted features and then aggregate the information of the context nodes; an adaptive weight module is set to distribute the output of the graph convolution module with attention weights, so as to better give the network an adaptive ability, thereby optimizing the output results and further improving the accuracy of sentiment classification.

[0187] The embodiment of the present invention further provides a terminal device, such as Figure 5 As shown, the terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 5 Only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100. When the processor D100 executes the computer program D102, the aspect-level sentiment analysis method based on the graph convolutional network is implemented.

[0188] The terminal device D10 can be a computing device such as a desktop computer, a notebook, a PDA, a server, a server cluster, a cloud server, etc. The terminal device may include, but is not limited to, a processor D100 and a memory D101. It will be understood by those skilled in the art that Figure 5 This is merely an example of the terminal device D10 and does not constitute a limitation on the terminal device D10 . The terminal device D10 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device D10 may also include input and output devices, network access devices, etc.

[0189] The processor D100 may be a central processing unit (CPU), or may be another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0190] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card, etc. equipped on the terminal device D10. Furthermore, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory D101 may also be used to temporarily store data that has been output or is to be output.

[0191] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0192] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0193] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements an aspect-level sentiment analysis method based on a graph convolutional network.

[0194] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to a construction device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk.

[0195] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for aspect-level sentiment analysis based on graph convolutional networks, characterized in that: include: Step 1: Get the text to be processed; Step 2: input the text to be processed into the trained aspect-level sentiment analysis model for analysis to obtain the sentiment polarity classification result corresponding to the text to be processed; The aspect-level sentiment analysis model includes an input module for converting the input text, an attention module for feature extraction, a graph convolution module for masking and aggregating the features output by the previous level, an adaptive weight module for weighting the aggregated features, and an output module for outputting the sentiment polarity classification results. The input end of the input module is the input end of the aspect-level sentiment analysis model; The output end of the input module is connected to the input end of the attention module; The output end of the attention module is connected to the input end of the graph convolution module and the first input end of the adaptive weight module; The first output terminal of the graph convolution module is connected to the second input terminal of the adaptive weight module; The second output end of the graph convolution module and the output end of the adaptive weight module are both connected to the input end of the output module; The output end of the output module is the output end of the aspect-level sentiment analysis model; Before inputting the text to be processed into the trained aspect-level sentiment analysis model for analysis, the method further includes: Preprocessing the text to be processed to obtain a preprocessed text; Inputting the preprocessed text into the trained aspect-level sentiment analysis model for analysis; The input module includes a conversion unit, an enhancement unit, and an encoding unit; Converting the preprocessed text into a plurality of word embedding vectors by the conversion unit; All word embedding vectors are combined with the introduced position embedding vector and part-of-speech embedding vector by the enhancement unit to obtain multiple enhanced vectors; Encoding all enhanced vectors by the encoding unit to generate a hidden state matrix, wherein the hidden state matrix includes multiple word vectors; The enhancement unit combines all word embedding vectors with the introduced position embedding vector and part-of-speech embedding vector to obtain multiple enhanced vectors: E i =Concatenate(x i ,p i ,t i ) Among them, E i represents the i-th enhancement vector, x i represents the i-th word embedding vector, p i represents the embedding vector of the i-th position, t i represents the i-th part-of-speech embedding vector, and Concatenate(·) represents the concatenation function.

2. The aspect-level sentiment analysis method based on graph convolutional network according to claim 1 is characterized in that The attention module includes an aspect-aware attention unit, a self-attention unit, and a merging unit; In the aspect-aware attention unit, the word vector extracted from the hidden state matrix is ​​converted into a query vector through a learning matrix, and the query vector is calculated through an aspect-aware attention function to obtain an aspect-aware attention weight, and the hidden state matrix is ​​weighted using the aspect-aware attention weight to obtain an aspect-aware feature matrix; In the self-attention unit, a query matrix, a key matrix, and a value matrix are obtained by combining multiple learning matrices with the hidden state matrix, the query matrix, the key matrix, and the value matrix are calculated using an activation function to obtain a self-attention weight matrix, and the self-attention weight matrix is ​​combined with the value matrix to obtain a self-attention perception feature matrix; The merging unit is used to merge the aspect perception feature matrix and the self-attention perception feature matrix to obtain a comprehensive perception feature matrix.

3. The aspect-level sentiment analysis method based on graph convolutional network according to claim 2 is characterized in that The graph convolution module includes a mask construction unit and a graph convolution unit; The mask construction unit is used to construct a mask matrix according to the syntactic dependency tree of the sentence, and the mask matrix is ​​used to perform mask processing on the comprehensive perception feature matrix; The graph convolution unit performs graph convolution processing on the comprehensive perception feature matrix after mask processing to obtain multiple node feature matrices.

4. The aspect-level sentiment analysis method based on graph convolutional network according to claim 3 is characterized in that The adaptive weight module is used to weight the aggregated features, including: Calculating a similarity score between the comprehensive perceptual feature matrix and each word vector; Calculate the attention weight of each word vector using the similarity score; Each of the node feature matrices is weighted and summed using the attention weight to obtain multiple weighted node feature matrices.

5. The aspect-level sentiment analysis method based on graph convolutional network according to claim 4 is characterized in that The output module includes an aggregation layer, a linear transformation layer and a nonlinear activation function layer; The first input terminal of the aggregation layer is connected to the first output terminal of the graph convolution unit, and the second terminal of the aggregation layer is connected to the output terminal of the adaptive weight module; The output end of the aggregation layer is connected to the input end of the linear transformation layer, the output end of the linear transformation layer is connected to the input end of the nonlinear activation function layer, and the output end of the nonlinear activation function layer is the output end of the aspect-level sentiment analysis model; The aggregation layer is used to aggregate the multiple node feature matrices and the multiple weighted node feature matrices in a series manner to obtain an aggregated feature matrix; The linear transformation layer is used to perform linear transformation on the aggregated feature matrix to obtain transformed features; The nonlinear activation function layer is used to perform sentiment polarity classification on the transformed features to obtain a predicted probability distribution corresponding to each sentiment category in the text to be processed.

6. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the aspect-level sentiment analysis method based on graph convolutional network as described in any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the aspect-level sentiment analysis method based on a graph convolutional network is implemented.

Images