Aspect-level sentiment analysis method and device based on graph convolutional network, and medium

Abstract

The invention provides an aspect-level sentiment analysis method and device based on a graph convolutional network and a medium, and the method comprises the steps: inputting a to-be-processed text into a trained aspect-level sentiment analysis model for analysis, and obtaining a sentiment polarity classification result corresponding to the to-be-processed text; the aspect-level sentiment analysis model comprises an input module, an attention module, a graph convolution module for masking and aggregating features output by a previous level, a self-adaptive weight module for weighting the aggregated features, and an output module; compared with the prior art, the method has the advantages that the graph convolution module is arranged in the aspect-level sentiment analysis model, and information of context nodes is aggregated after mask processing is performed on extracted features, so that more attention can be paid to key parts contributing to sentiment analysis; and the adaptive weight module is set, and the output of the graph convolution module is subjected to attention weight distribution, so that the network is better endowed with adaptive capability, the output result is optimized, and the sentiment classification accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing and artificial intelligence technology, and in particular to an aspect-level sentiment analysis method, device and medium based on a graph convolutional network. Background Art

[0002] With the rise of social media and online platforms, the amount of content generated has grown exponentially, and these contents contain rich emotional and opinion information. Sentiment analysis is also generally called opinion mining, which is used to analyze the emotional polarity, emotions, evaluations, attitudes, and other factors that humans have towards a target.

[0003] In the aspect-level sentiment analysis task, the evolution of research methods has gradually transitioned from the early traditional methods based on rules and dictionaries to deep learning technology through the machine learning stage. Early research mainly focused on using sentiment dictionaries or methods based on specific rules to determine sentiment polarity. For example, some researchers compared the performance of naive Bayes, maximum entropy models, and support vector machines in sentiment classification tasks and found that support vector machines showed better results. Some researchers also proposed a dictionary-based method to extract features specifically for explicit and implicit sentiment polarity in texts.

[0004] With the development of deep learning technology, aspect-level sentiment analysis tasks have begun to use more complex neural network models for sentiment analysis, such as recurrent neural networks and their variants, convolutional neural networks, etc. These models can capture the sequence characteristics and local context information of texts, thereby improving the accuracy of sentiment analysis. For example, some researchers have proposed a recursive neural network based on dependency trees, which can adaptively transmit emotions based on the syntactic relationship and context information between words, effectively improving the performance of sentiment classification; other researchers have combined bidirectional long short-term memory networks and support vector regression models to propose a composite model for fine-grained sentiment analysis tasks.

[0005] In recent years, graph neural networks, especially graph convolutional networks and graph attention networks, have been widely used in aspect-level sentiment analysis due to their advantages in processing graph-structured data. These graph-based models effectively integrate syntactic dependency information and contextual features by constructing dependency graphs between words, further improving the performance of sentiment classification. For example, some researchers have proposed a model that combines an aspect-based attention mechanism with a graph convolutional neural network, focusing on the fusion of aspect-level features and contextual information; some researchers have also designed a graph convolutional neural network model that combines a special gating mechanism with a self-attention network, which can dynamically integrate information from word dependency graphs and latent graphs.

[0006] Although traditional methods and deep learning-based models have been successful in specific scenarios, they have limitations in capturing deep semantic relationships and subtle sentiment differences in complex text structures, resulting in low accuracy of sentiment classification results. Summary of the invention

[0007] The present invention provides an aspect-level sentiment analysis method, device and medium based on a graph convolutional network, the purpose of which is to improve the accuracy of sentiment classification.

[0008] In order to achieve the above object, the present invention provides an aspect-level sentiment analysis method based on a graph convolutional network, comprising:

[0009] Step 1, obtain the text to be processed;

[0010] Step 2: Input the text to be processed into the trained aspect-level sentiment analysis model for analysis to obtain the sentiment polarity classification result corresponding to the text to be processed;

[0011] The aspect-level sentiment analysis model includes an input module for converting input text, an attention module for feature extraction, a graph convolution module for masking and aggregating features output from the previous level, an adaptive weight module for weighting the aggregated features, and an output module for outputting sentiment polarity classification results;

[0012] The input end of the input module is the input end of the aspect-level sentiment analysis model;

[0013] The output of the input module is connected to the input of the attention module;

[0014] The output end of the attention module is connected to the input end of the graph convolution module and the first input end of the adaptive weight module;

[0015] The first output terminal of the graph convolution module is connected to the second input terminal of the adaptive weight module;

[0016] The second output terminal of the graph convolution module and the output terminal of the adaptive weight module are both connected to the input terminal of the output module;

[0017] The output end of the output module is the output end of the aspect-level sentiment analysis model.

[0018] Furthermore, before the text to be processed is input into the trained aspect-level sentiment analysis model for analysis, the following steps are also included:

[0019] Preprocess the text to be processed to obtain the preprocessed text;

[0020] The preprocessed text is input into the trained aspect-level sentiment analysis model for analysis.

[0021] Further, the input module includes a conversion unit, an enhancement unit, and an encoding unit;

[0022] The preprocessed text is converted into multiple word embedding vectors through the conversion unit;

[0023] All word embedding vectors are combined with the introduced position embedding vector and part-of-speech embedding vector through the enhancement unit to obtain multiple enhanced vectors;

[0024] All enhanced vectors are encoded through the encoding unit to generate a hidden state matrix, which includes multiple word vectors.

[0025] Furthermore, all word embedding vectors are combined with the introduced position embedding vector and part-of-speech embedding vector through the enhancement unit, and the expressions of multiple enhanced vectors are obtained as follows:

[0026] E i =Concatenate(x i ,p i ,t i )

[0027] Among them, E i represents the i-th enhancement vector, x i represents the i-th word embedding vector, p i represents the embedding vector of the i-th position, t i represents the i-th part-of-speech embedding vector, and COncatenate(·) represents the connection function.

[0028] Further, the attention module includes aspect-aware attention unit, self-attention unit and merging unit;

[0029] In the aspect-aware attention unit, the word vector extracted from the hidden state matrix is ​​converted into a query vector through the learning matrix, and the query vector is calculated through the aspect-aware attention function to obtain the aspect-aware attention weight. The hidden state matrix is ​​weighted by the aspect-aware attention weight to obtain the aspect-aware feature matrix.

[0030] In the self-attention unit, multiple learning matrices are combined with the hidden state matrix to obtain the query matrix, key matrix and value matrix. The query matrix, key matrix and value matrix are calculated through the activation function to obtain the self-attention weight matrix, and the self-attention weight matrix is ​​combined with the value matrix to obtain the self-attention perception feature matrix.

[0031] The merging unit is used to merge the aspect perception feature matrix and the self-attention perception feature matrix to obtain a comprehensive perception feature matrix.

[0032] Further, the graph convolution module includes a mask construction unit and a graph convolution unit;

[0033] The mask construction unit is used to construct a mask matrix according to the syntactic dependency tree of the sentence, and the mask matrix is ​​used to perform mask processing on the comprehensive perception feature matrix;

[0034] The graph convolution unit performs graph convolution on the comprehensive perception feature matrix after mask processing to obtain multiple node feature matrices.

[0035] Specifically, the adaptive weight module is used to weight the aggregated features through the aspect-level attention mechanism, including:

[0036] Calculate the similarity score between the comprehensive perceptual feature matrix and each word vector;

[0037] Use the similarity score to calculate the attention weight of each word vector;

[0038] The feature matrix of each node is weighted and summed by the attention weight to obtain multiple weighted node feature matrices.

[0039] Further, the output module includes an aggregation layer, a linear transformation layer, and a nonlinear activation function layer;

[0040] A first input terminal of the aggregation layer is connected to a first output terminal of the graph convolution unit, and a second terminal of the aggregation layer is connected to an output terminal of the adaptive weight module;

[0041] The output end of the aggregation layer is connected to the input end of the linear transformation layer, the output end of the linear transformation layer is connected to the input end of the nonlinear activation function layer, and the output end of the nonlinear activation function layer is the output end of the aspect-level sentiment analysis model;

[0042] The aggregation layer is used to aggregate multiple node feature matrices and multiple weighted node feature matrices in series to obtain an aggregated feature matrix;

[0043] The linear transformation layer is used to perform linear transformation on the aggregated feature matrix to obtain the transformed features;

[0044] The nonlinear activation function layer is used to classify the transformed features into sentiment polarity and obtain the predicted probability distribution corresponding to each sentiment category in the text to be processed.

[0045] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, an aspect-level sentiment analysis method based on a graph convolutional network is implemented.

[0046] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements an aspect-level sentiment analysis method based on a graph convolutional network.

[0047] The above scheme of the present invention has the following beneficial effects:

[0048] The present invention obtains the sentiment polarity classification result corresponding to the text to be processed by analyzing the acquired text to be processed input into the trained aspect-level sentiment analysis model; the aspect-level sentiment analysis model comprises an input module for converting the input text, an attention module for feature extraction, a graph convolution module for masking and aggregating the features outputted at the previous level, an adaptive weight module for weighting the aggregated features, and an output module for outputting the sentiment polarity classification result; compared with the prior art, the present invention can pay more attention to the key parts contributing to the sentiment analysis by arranging a graph convolution module in the aspect-level sentiment analysis model to mask the extracted features and then aggregate the information of the context nodes; an adaptive weight module is arranged to distribute the output of the graph convolution module with attention weights, so as to better endow the network with adaptive ability, thereby optimizing the output result, and further improving the accuracy of sentiment classification.

[0049] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of a flow chart of an embodiment of the present invention;

[0051] Figure 2 Schematic diagram of the structure of the aspect-level sentiment analysis model in an embodiment of the present invention;

[0052] Figure 3 The effect of different numbers of graph convolution units on the accuracy results of two data sets in an embodiment of the present invention;

[0053] Figure 4 This is the effect of different numbers of graph convolutional units on the macro-average F1 results on two data sets in an embodiment of the present invention;

[0054] Figure 5 A schematic diagram of the structure of a terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0057] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a locking connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0058] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0059] In view of the existing problems, the present invention provides an aspect-level sentiment analysis method, device and medium based on graph convolutional network.

[0060] like Figure 1 As shown, an embodiment of the present invention provides an aspect-level sentiment analysis method based on a graph convolutional network, comprising:

[0061] Step 1, obtaining the text to be processed, that is, the original text to be used for sentiment analysis, such as the original dialogue text of the review software;

[0062] Step 2: Input the text to be processed into the trained aspect-level sentiment analysis model for analysis to obtain the sentiment polarity classification result corresponding to the text to be processed. The sentiment polarity can be positive, negative or neutral.

[0063] The aspect-level sentiment analysis model includes an input module for converting input text, an attention module for feature extraction, a graph convolution module for aggregating features output from the previous level, an adaptive weight module for weighting the aggregated features, and an output module for outputting sentiment polarity classification results;

[0064] The input end of the input module is the input end of the aspect-level sentiment analysis model;

[0065] The output of the input module is connected to the input of the attention module;

[0066] The output end of the attention module is connected to the input end of the graph convolution module and the first input end of the adaptive weight module;

[0067] The first output terminal of the graph convolution module is connected to the second input terminal of the adaptive weight module;

[0068] The second output terminal of the graph convolution module and the output terminal of the adaptive weight module are both connected to the input terminal of the output module;

[0069] The output end of the output module is the output end of the aspect-level sentiment analysis model.

[0070] The specific process of model processing is as follows:

[0071] Input the original text, which is the original dialogue text of the review software, such as "The scenery of this place is so beautiful". After the embedding layer, a preliminary semantic vector representation is obtained; after the attention layer, the original attention matrix is ​​obtained; the original attention matrix is ​​processed by the syntactic mask matrix to obtain a representation matrix with enhanced semantics, and then the enhanced matrix is ​​obtained by the node information aggregation method of the graph convolution layer; the enhanced matrix and the original attention matrix are weighted to obtain a weight matrix to integrate the original information; finally, the weight matrix and the enhanced matrix are merged in dimension to obtain the final output matrix, and then after a series of operations of the activation function, the final sentiment analysis prediction probability distribution is obtained, and the final sentiment analysis prediction distribution is positive sentiment polarity.

[0072] Specifically, before the text to be processed is input into the trained aspect-level sentiment analysis model for analysis, the following steps are also included:

[0073] Preprocess the text to be processed to obtain the preprocessed text;

[0074] The preprocessed text is input into the trained aspect-level sentiment analysis model for analysis.

[0075] In an embodiment of the present invention, preprocessing is used to clean the text to be processed into a format suitable for analysis, specifically including removing useless information (such as HTML tags), performing word form restoration, standardizing the text format (such as converting plural words into basic forms, converting all text to be processed into lowercase), etc.

[0076] The input module includes a conversion unit, an enhancement unit, and an encoding unit;

[0077] The preprocessed text is converted into multiple word embedding vectors through the conversion unit;

[0078] All word embedding vectors are combined with the introduced position embedding vector and part-of-speech embedding vector through the enhancement unit to obtain multiple enhanced vectors;

[0079] All enhanced vectors are encoded through the encoding unit to generate a hidden state matrix, which includes multiple word vectors.

[0080] In the embodiment of the present invention, the conversion unit is internally integrated with a pre-trained word embedding model such as Word2Vec or GloVe or FastText or BERT or ELMo, which is used to convert each word w in the pre-processed text into i Are mapped to a high-dimensional vector space to obtain word embedding vectors where d e Represents the dimension of word embedding vectors. These vectors can capture the semantic information of words. Therefore, the word embedding vectors of the entire text can form an embedding matrix

[0081] In order to further enrich the representation of word vectors, the embodiment of the present invention introduces position embedding vectors and part-of-speech embedding vectors in the enhancement unit, and combines these embedding vectors with the embedding matrix Each word embedding vector in is combined to obtain multiple enhanced vectors, so that the enhanced vector of the i-th word embedding vector can be calculated by the following expression:

[0082] E i =Concatenate(x i ,p i ,t i )

[0083] Among them, E i represents the i-th enhancement vector, x i represents the i-th word embedding vector, p i represents the embedding vector of the i-th position, t i represents the i-th part-of-speech embedding vector, and Concatenate(·) represents the concatenation function.

[0084] In order to capture context information, the embodiment of the present invention applies a bidirectional long short-term memory network (Bi-LSTM) in the encoding unit to encode the enhanced vector to generate a series of context-sensitive word vectors To form the hidden state matrix where d h represents the hidden layer dimension of the bidirectional long short-term memory network, n represents the number of words in the sentence, and represents the dimension of the hidden state.

[0085] Most preferably, the attention module comprises an aspect-aware attention unit, a self-attention unit, and a merging unit;

[0086] In the aspect-aware attention unit, the word vector extracted from the hidden state matrix is ​​converted into a query vector through the learning matrix, and the query vector is calculated through the aspect-aware attention function to obtain the aspect-aware attention weight. The hidden state matrix is ​​weighted by the aspect-aware attention weight to obtain the aspect-aware feature matrix.

[0087] In the self-attention unit, multiple learning matrices are combined with the hidden state matrix to obtain the query matrix, key matrix and value matrix. The query matrix, key matrix and value matrix are calculated through the activation function to obtain the self-attention weight matrix, and the self-attention weight matrix is ​​combined with the value matrix to obtain the self-attention perception feature matrix.

[0088] The merging unit is used to merge the aspect perception feature matrix and the self-attention perception feature matrix to obtain a comprehensive perception feature matrix.

[0089] Specifically, the aspect-aware attention mechanism focuses on the sentiment expression of specific aspect terms. Its design is based on the following mathematical formalization, and the aspect-aware attention unit is specifically used for:

[0090] Extract the word vector h associated with the aspect term from the hidden state matrix H by position index a ;

[0091] In order to capture the context information related to aspect terms, the embodiment of the present invention introduces a learnable learning matrix

[0092] By learning the matrix W a The extracted word vector h a Convert to query vector q a , the conversion expression is:

[0093] q a =h a W a

[0094] The query vector q is processed by the aspect-aware attention function. a Calculate and get the aspect-aware attention weight A asp , the aspect-aware attention weight is used to characterize the contribution of each word in the sentence to a specific aspect term, and the aspect-aware attention function is:

[0095] A asp =Softmax(q a K T )

[0096] Among them, K represents the key matrix, T represents the transposed matrix, and the Softmax activation function is used to normalize the attention weights;

[0097] Using the aspect-aware attention weight A asp The hidden state matrix H is weighted to obtain the aspect perception feature H for predicting the sentiment tendency of aspect terms. asp , the weighted expression is:

[0098] H asp =A asp H.

[0099] Specifically, the self-attention mechanism captures the complex relationship within a sentence by calculating the influence weight of each word in the sentence on all other words. This mechanism allows the model to adaptively focus on the key parts of the sentence to obtain richer semantic representations. The self-attention unit is specifically used for:

[0100] Use different learning matrices to combine with the hidden state matrix to generate the query matrix Q, key matrix K and value matrix V respectively:

[0101] The query matrix is: Q = HW Q ;

[0102] The bond matrix is: K = HW K ;

[0103] The value matrix is: V = HW V ;

[0104] Among them, W Q , W K , W V All represent learnable learning matrices;

[0105] Based on the activation function, the self-attention weight matrix A is calculated by the query matrix Q, the key matrix K and the value matrix V as:

[0106]

[0107] Among them, d k Indicates the dimension of the key vector, which is used to scale the result of the dot product to prevent the gradient from disappearing due to excessive inner product values;

[0108] Combine the self-attention weight matrix with the value matrix to obtain the self-attention perception feature matrix H self , the expression is:

[0109] H self =AV

[0110] Among them, A represents the self-attention weight matrix and V represents the value matrix.

[0111] The embodiment of the present invention combines the aspect perception feature matrix and the self-attention perception feature matrix through a merging unit to obtain a comprehensive perception feature matrix H att , the expression is:

[0112] H att =H asp +H self

[0113] The model that integrates the aspect-aware attention feature matrix and the self-attention feature matrix can capture aspect-related information while making full use of the lexical relations within the sentence.

[0114] In order to further enhance the model's understanding of the syntactic structure of sentences, an embodiment of the present invention introduces a syntactic mask in the graph convolution module for use in combination with the graph convolution. This design utilizes the syntactic dependency information of the sentence and guides the graph convolution to capture and utilize the dependency relationship between words through the syntactic mask, thereby achieving effective processing of complex sentence structures.

[0115] Most preferably, the graph convolution module comprises a mask construction unit and a graph convolution unit;

[0116] The mask construction unit is used to construct a mask matrix according to the syntactic dependency tree of the sentence, and the mask matrix is ​​used to perform mask processing on the comprehensive perception feature matrix;

[0117] The graph convolution unit performs graph convolution on the comprehensive perception feature matrix after mask processing to obtain multiple node feature matrices.

[0118] In the embodiment of the present invention, the syntactic mask is constructed according to the syntactic dependency tree of the sentence. For any two word pairs in the sentence, a syntactic mask matrix is ​​defined. The distance between the word pairs in the syntactic dependency tree represents the syntactic connection strength between the words. The construction process of the matrix is ​​as follows:

[0119] Construct a matrix and initialize it to an all-zero matrix;

[0120] For word pairs that have direct syntactic dependency in the sentence, the sum is set to 1, indicating that there is direct syntactic dependency between the two words;

[0121] For indirectly dependent word pairs, the mask value is adjusted according to their distance in the syntactic dependency tree. The shortest syntactic path length between each pair of words is calculated, and then the syntactic mask matrix M is constructed based on this distance. The calculation expression of the shortest syntactic path length is:

[0122]

[0123] Where k is a predefined syntactic distance threshold used to determine which syntactic connections between words are strong enough to be considered, and dij represents the binary weight assigned to each word pair, indicating whether they are syntactically dependent.

[0124] The graph convolution unit takes the hidden state matrix processed by the syntactic mask matrix as input, and aggregates the input information through the graph convolution operation. The expression of the graph convolution operation is:

[0125]

[0126] in, represents the node feature matrix of the lth layer in the graph convolution unit, Represents the syntactic mask matrix M and the identity matrix I n The sum of is used to ensure that each node’s own features are also included. represents a diagonal matrix whose elements yes The sum of the i-th row in is used for normalization, W (l) represents the learning matrix of the lth layer, σ represents the nonlinear activation function;

[0127] Through the above calculations, each node not only obtains its own feature information, but also integrates the features of the nodes directly connected to it in the syntactic tree. This integrated information can improve the accuracy of sentiment analysis. Multi-layer graph convolution units can learn the deep syntactic characteristics of sentences and enrich the semantic representation of sentences. After further processing by the graph convolution unit, multiple more accurate node feature matrices are obtained.

[0128] Specifically, the adaptive weight module is used to weight the aggregated features, including:

[0129] Calculate the similarity score between the comprehensive perceptual feature matrix and each word vector;

[0130] Use the similarity score to calculate the attention weight of each word vector;

[0131] The feature matrix of each node is weighted and summed by the attention weight to obtain multiple weighted node feature matrices.

[0132] Specifically, in the aspect-level attention mechanism, what is actually represented is the weight of the importance of each word to a specific aspect. These weights are calculated by comparing the similarity between the representation of each word in the sentence and the aspect representation. Therefore, for each word vector, the embodiment of the present invention calculates the similarity score between it and the comprehensive perceptual feature matrix, which is expressed as:

[0133] S i =H att T H i

[0134] Among them, S i represents the similarity score;

[0135] The similarity score is used to calculate the attention weight of each word, ensuring that the sum of all weights is 1. In this way, the weight of each word represents its importance in terms of attention. The calculation expression is:

[0136]

[0137] Among them, W asp represents the attention weight, and N represents the total number of words:

[0138] After obtaining the attention weights, in order to further emphasize the words related to specific aspects, the node feature matrix output by the graph convolution unit is Weighted to form weighted output, the process can be expressed by the following formula:

[0139]

[0140] Among them, O weighted represents the i-th weighted node feature matrix, W asp,i represents the attention weight of the i-th word.

[0141] Most preferably, the output module includes an aggregation layer, a linear transformation layer, and a nonlinear activation function layer;

[0142] A first input terminal of the aggregation layer is connected to a first output terminal of the graph convolution unit, and a second terminal of the aggregation layer is connected to an output terminal of the adaptive weight module;

[0143] The output end of the aggregation layer is connected to the input end of the linear transformation layer, the output end of the linear transformation layer is connected to the input end of the nonlinear activation function layer, and the output end of the nonlinear activation function layer is the output end of the aspect-level sentiment analysis model;

[0144] The aggregation layer is used to aggregate multiple node feature matrices and multiple weighted node feature matrices in series to obtain an aggregated feature matrix;

[0145] The linear transformation layer is used to perform linear transformation on the aggregated feature matrix to obtain the transformed features;

[0146] The nonlinear activation function layer is used to classify the transformed features into sentiment polarity and obtain the predicted probability distribution corresponding to each sentiment category in the text to be processed.

[0147] Specifically, in order to utilize these multi-level information, the node feature matrices output by all layers are aggregated with multiple weighted node feature matrices in series in the aggregation layer. The aggregation expression is:

[0148]

[0149] Among them, O combined represents the aggregated feature matrix, Concat(·) represents the concatenation operation on the feature dimension, Represents the node feature matrix of the Lth layer in the graph convolution unit;

[0150] Before finally mapping the comprehensive output to the emotion category, in order to reduce the risk of overfitting and enhance the generalization ability of the model, the aggregated feature matrix is ​​then linearly transformed through a linear transformation layer to obtain the transformed feature O output , the expression is:

[0151] O output =ReLU(dropout(W b ·O combined +b))

[0152] Among them, W b , b represent the weight and bias of the linear transformation layer, respectively. Dropout(·) is used to prevent overfitting. ReLU(·) is a nonlinear activation function.

[0153] Then, a nonlinear activation function is applied to increase the expressive power of the model. The nonlinear activation function layer is used to classify the transformed features into sentiment polarity and obtain the predicted probability distribution corresponding to each sentiment category in the text to be processed. The expression is:

[0154] P = softmax(W f O output +b f )

[0155] Among them, W f represents the weight of the nonlinear activation function softmax, b f represents the bias term, and P represents the predicted probability distribution corresponding to each sentiment category in the text to be processed.

[0156] In the embodiment of the present invention, for each text, the loss function calculates the difference between the probability distribution of the model output and the distribution of the true label. Assuming that the output of the model for a certain text is a probability distribution, where the total number of categories is the probability that the model predicts that the current text belongs to the category, if the true label is (represented by one-hot encoding, that is, 1 in the correct category position and 0 in other positions), then the cross entropy loss can be expressed as:

[0157]

[0158] Among them, y iis the value of the true label at the position of category i. If the current text truly belongs to category i, then y i =1, otherwise y i =0.

[0159] During the training process, the model parameters are adjusted by minimizing this loss function so that the model's output P becomes closer and closer to the true label distribution y. In this way, the model can learn how to correctly classify text. In addition, for BERT-based models, regularization terms are introduced to alleviate the overfitting problem, such as using Dropout in BERT's attention mechanism and hidden layers. These techniques help improve the model's generalization ability on unseen data, and ultimately obtain a trained aspect-level sentiment analysis model.

[0160] The embodiment of the present invention conducts experiments on aspect-based sentiment analysis on three benchmark datasets. The benchmark datasets include Restaurant and Laptop computer reviews and Twitter articles from SemEval 2014 Task 4. Each aspect has one of three sentiment polarities: positive, neutral, and negative. The statistical data of the three datasets are shown in Table 1 below:

[0161] Table 1

[0162]

[0163] To initialize the conversion unit, the embodiment of the present invention uses pre-trained GloVe vectors, which are learned from a large text corpus and have a dimension of 300, providing rich semantic information for the model. In addition, the model uses a bidirectional long short-term memory network integrated into the encoding unit after the conversion unit to capture the context information in the sentence, and the hidden state dimension in each direction is set to 50.

[0164] During the training process, the embodiment of the present invention uses the Adam optimizer and sets a learning rate of 0.002 to promote the convergence of the model. The batch size is set to 16, allowing the model to process more data in each training, while also maintaining a certain batch size to prevent overfitting. The dimension of the position embedding vector is set to 30 to help the model better understand the relative position of the word in the sentence.

[0165] The model of the embodiment of the present invention includes a graph convolution unit composed of a graph convolutional network, which uses a dimension of 768 to encode the context information of each word. In order to further improve the model performance, a 5-head multi-head attention mechanism is adopted, and a dropout rate of 0.2 is set to avoid overfitting. In order to ensure the reproducibility of the experimental results, the random seed in the experiment is fixed at 1000, and the BERT pre-trained model is additionally used for parameter initialization, providing a powerful semantic representation foundation for the model.

[0166] In order to comprehensively evaluate the model of the embodiment of the present invention, the embodiment of the present invention selected the following classic baseline models for comparison:

[0167] RAM uses bidirectional LSTM and attention mechanism to learn sentence representation;

[0168] TNet uses a CNN model to extract salient features from target-specific embeddings through converted Bi-LSTM embeddings;

[0169] ASGCN constructs aspect attention and combines it with GCN to extract aspect features of sentences;

[0170] CDT uses convolution on the dependency tree model to learn the representation of sentence features;

[0171] Bi-GCN proposes a global vocabulary graph to encode word co-occurrence information, and at the same time establishes a concept hierarchy on the local syntactic graph and vocabulary graph to distinguish different types of co-occurrence relationships;

[0172] KumaGCN combines information from dependency graph and latent graph to learn grammatical features;

[0173] DGEDT proposes a graph-enhanced Transformer network, introducing dual affine transformations to interactively learn features from Transformer and graph-based information from GCN.

[0174] Baseline Model Results and Analysis

[0175] The model provided by the embodiment of the present invention and the baseline model are compared in terms of aspect-level sentiment analysis accuracy and macro-average F1 on three public datasets. The results are shown in Table 2:

[0176] Table 2

[0177]

[0178]

[0179] As shown in Table 2, on the Laptop dataset, the SM-GCN model of the embodiment of the present invention leads with an accuracy of 79.43% and a macro-average F1 score of 76.38%, showing a clear advantage over other baseline models. This excellent performance highlights the effectiveness of the SM-GCN model of the embodiment of the present invention in capturing specific emotional information; for the Restaurant dataset, the model of the embodiment of the present invention also leads in accuracy (83.82%) and F1 score (77.02%). Compared with the Bi-GCN model, the GCN of the embodiment of the present invention has an improvement of nearly 2% in accuracy, which shows that the model of the embodiment of the present invention has better performance when processing user comment data containing complex emotional expressions. On the Twitter dataset, although the text characteristics of the dataset are more informal and diverse, the model of the embodiment of the present invention still achieves a macro-average F1 score of 73.78%, showing the robustness of the model for text with more casual structures. For the BERT-based model, in short, the BERT model shows higher performance on all datasets. In particular, on the Twitter dataset, the model of the embodiment of the present invention has a significant improvement. However, on the Restaurant dataset, the DGEDT+BERT model is slightly ahead with a macro-average F1 score of 80.00%, which may be due to the special applicability of the combination of a specific model architecture and BERT in this field. Nevertheless, the model of the embodiment of the present invention still demonstrates the potential of deep semantic analysis combined with graph convolutional networks in improving the results of sentiment classification tasks.

[0180] In order to gain a deeper understanding of the contribution of each component in the model, the embodiment of the present invention conducted a series of ablation experiments and removed the key components of the model: attention module, syntactic mask matrix, and adaptive weight module. The removed components are marked with "w / o". The specific experimental results are shown in Table 3:

[0181] Table 3

[0182]

[0183] First, when the attention module is removed, the accuracy of the model on the Laptop dataset drops from 79.43% to 75.74%, and the F1 score drops from 76.38% to 70.98%. This significant performance drop indicates that the attention mechanism is essential for capturing contextual information related to specific aspects. Similarly, the performance on the Restaurant dataset also has a similar downward trend, with the accuracy dropping from 83.82% to 82.39% and the F1 score dropping from 77.02% to 74.14%. Removing the syntactic mask matrix causes a slight drop in the performance of the model on the Laptop and Restaurant datasets, respectively, while the drop on the Twitter dataset is more obvious, from 75.33% accuracy to 75.04%, and the F1 score slightly dropping from 73.78% to 73.40%. This shows that the impact of syntactic structure on the model varies depending on the characteristics of the dataset. For some datasets, the lack of syntactic information will have a negative impact on the model's understanding of text structure and semantics. The impact of eliminating the adaptive weight module on model performance is also not negligible. Especially on the Twitter dataset, the accuracy of the model dropped from 75.33% to 73.18%, and the F1 score dropped from 73.78% to 71.18%, which reveals the important role of attention weights in capturing emotional tendencies and aspect-related details. In summary, the ablation experiment shows that each module contributes to the overall performance of the model.

[0184] Finally, we explore the impact of the number of GCN layers on the results

[0185] In order to explore the impact of different numbers of GCN layers on the results, the embodiment of the present invention sets the number of GCN layers to 1 to 4 in two data sets, and analyzes the impact of different numbers of layers on the accuracy and macro-average F1 results. Figure 3 , as shown in 4. When exploring the impact of the number of GCN layers on the results, the embodiment of the present invention conducted a detailed analysis of the results of the two data sets, Restaurant and Twitter, which have relatively complex data structures. For the Restaurant data set, experiments show that the configuration of a single-layer GCN can achieve optimal performance. As the number of GCN layers increases, it can be noticed that the performance of the model shows a downward trend. In particular, when the number of layers increases to four, the performance degradation is particularly obvious. This may be because the model begins to have the problem of over-smoothing, which makes the distinction between different nodes lower. On the Twitter data set, the performance of the model under the single-layer GCN structure is also the best, and the increase in the number of layers will lead to a decline in performance, especially when the number of layers reaches four, the performance loss is significant. The embodiment of the present invention can see that in this type of task, too many layers may cause the model to be unable to effectively capture useful features, but instead damage the generalization ability of the model.

[0186] The embodiment of the present invention obtains the sentiment polarity classification result corresponding to the text to be processed by analyzing the acquired text to be processed input into the trained aspect-level sentiment analysis model; the aspect-level sentiment analysis model includes an input module for converting the input text, an attention module for feature extraction, a graph convolution module for masking and aggregating the features outputted at the previous level, an adaptive weight module for weighting the aggregated features, and an output module for outputting the sentiment polarity classification result; compared with the prior art, the embodiment of the present invention can pay more attention to the key parts that contribute to the sentiment analysis by setting a graph convolution module in the aspect-level sentiment analysis model to mask the extracted features and then aggregate the information of the context nodes; an adaptive weight module is set to distribute the output of the graph convolution module with attention weights, so as to better give the network the ability to adapt, thereby optimizing the output results, and further improving the accuracy of sentiment classification.

[0187] The embodiment of the present invention also provides a terminal device, such as Figure 5 As shown, the terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 5 Only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100. When the processor D100 executes the computer program D102, the aspect-level sentiment analysis method based on the graph convolutional network is implemented.

[0188] The terminal device D10 may be a computing device such as a desktop computer, a notebook, a PDA, a server, a server cluster, a cloud server, etc. The terminal device may include, but is not limited to, a processor D100 and a memory D101. Those skilled in the art will appreciate that Figure 5 This is only an example of the terminal device D10 and does not constitute a limitation on the terminal device D10. The terminal device D10 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0189] The processor D100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0190] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc. equipped on the terminal device D10. Further, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or is to be output.

[0191] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0192] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0193] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements an aspect-level sentiment analysis method based on a graph convolutional network.

[0194] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the construction device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0195] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for aspect-level sentiment analysis based on graph convolutional networks, characterized in that: include: Step 1, obtain the text to be processed; Step 2, inputting the text to be processed into the trained aspect-level sentiment analysis model for analysis to obtain a sentiment polarity classification result corresponding to the text to be processed; The aspect-level sentiment analysis model includes an input module for converting input text, an attention module for feature extraction, a graph convolution module for masking and aggregating features output from the previous level, an adaptive weight module for weighting the aggregated features, and an output module for outputting sentiment polarity classification results; The input end of the input module is the input end of the aspect-level sentiment analysis model; The output end of the input module is connected to the input end of the attention module; The output end of the attention module is connected to the input end of the graph convolution module and the first input end of the adaptive weight module; The first output terminal of the graph convolution module is connected to the second input terminal of the adaptive weight module; The second output end of the graph convolution module and the output end of the adaptive weight module are both connected to the input end of the output module; The output end of the output module is the output end of the aspect-level sentiment analysis model.

2. The aspect-level sentiment analysis method based on graph convolutional network according to claim 1, characterized in that: Before inputting the to-be-processed text into the trained aspect-level sentiment analysis model for analysis, the method further includes: Preprocessing the text to be processed to obtain a preprocessed text; The preprocessed text is input into the trained aspect-level sentiment analysis model for analysis.

3. The aspect-level sentiment analysis method based on graph convolutional network according to claim 2 is characterized in that: The input module includes a conversion unit, an enhancement unit, and an encoding unit; Converting the preprocessed text into a plurality of word embedding vectors by the conversion unit; The enhancement unit combines all word embedding vectors with the introduced position embedding vector and part-of-speech embedding vector to obtain multiple enhanced vectors; All enhanced vectors are encoded by the encoding unit to generate a hidden state matrix, where the hidden state matrix includes multiple word vectors.

4. The aspect-level sentiment analysis method based on graph convolutional network according to claim 3 is characterized in that: The enhancement unit combines all word embedding vectors with the introduced position embedding vector and part-of-speech embedding vector to obtain multiple enhanced vectors: E i =Concatenate(x i ,p i ,t i ) Among them, E i represents the i-th enhancement vector, x i represents the i-th word embedding vector, p i represents the embedding vector of the i-th position, t i represents the i-th part-of-speech embedding vector, and Concatenate(·) represents the concatenation function.

5. The aspect-level sentiment analysis method based on graph convolutional network according to claim 4 is characterized in that: The attention module includes an aspect-aware attention unit, a self-attention unit, and a merging unit; In the aspect-aware attention unit, the word vector extracted from the hidden state matrix is ​​converted into a query vector through a learning matrix, and the query vector is calculated through an aspect-aware attention function to obtain an aspect-aware attention weight, and the hidden state matrix is ​​weighted by the aspect-aware attention weight to obtain an aspect-aware feature matrix; In the self-attention unit, a query matrix, a key matrix and a value matrix are obtained by combining a plurality of learning matrices with the hidden state matrix, the query matrix, the key matrix and the value matrix are calculated by an activation function to obtain a self-attention weight matrix, and the self-attention weight matrix is ​​combined with the value matrix to obtain a self-attention perception feature matrix; The merging unit is used to merge the aspect perception feature matrix and the self-attention perception feature matrix to obtain a comprehensive perception feature matrix.

6. The aspect-level sentiment analysis method based on graph convolutional network according to claim 5, characterized in that: The graph convolution module includes a mask construction unit and a graph convolution unit; The mask construction unit is used to construct a mask matrix according to the syntactic dependency tree of the sentence, and the mask matrix is ​​used to perform mask processing on the comprehensive perceptual feature matrix; The graph convolution unit performs graph convolution processing on the comprehensive perception feature matrix after mask processing to obtain multiple node feature matrices.

7. The aspect-level sentiment analysis method based on graph convolutional network according to claim 6, characterized in that: The adaptive weight module is used to weight the aggregated features, including: Calculating a similarity score between the comprehensive perceptual feature matrix and each word vector; Calculate the attention weight of each word vector using the similarity score; Each of the node feature matrices is weighted and summed using the attention weight to obtain a plurality of weighted node feature matrices.

8. The aspect-level sentiment analysis method based on graph convolutional network according to claim 7, characterized in that: The output module includes an aggregation layer, a linear transformation layer and a nonlinear activation function layer; A first input terminal of the aggregation layer is connected to a first output terminal of the graph convolution unit, and a second terminal of the aggregation layer is connected to an output terminal of the adaptive weight module; The output end of the aggregation layer is connected to the input end of the linear transformation layer, the output end of the linear transformation layer is connected to the input end of the nonlinear activation function layer, and the output end of the nonlinear activation function layer is the output end of the aspect-level sentiment analysis model; The aggregation layer is used to aggregate the multiple node feature matrices and the multiple weighted node feature matrices in series to obtain an aggregated feature matrix; The linear transformation layer is used to perform linear transformation on the aggregated feature matrix to obtain transformed features; The nonlinear activation function layer is used to perform sentiment polarity classification on the transformed features to obtain a predicted probability distribution corresponding to each sentiment category in the text to be processed.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the aspect-level sentiment analysis method based on graph convolutional network is implemented as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the aspect-level sentiment analysis method based on a graph convolutional network as described in any one of claims 1 to 8 is implemented.

Images