An aspect-level sentiment analysis method based on a space-time graph attention network

Abstract

The application discloses an aspect-level sentiment analysis method based on a space-time graph attention network, wherein the method learns a hidden vector of a review text through a bidirectional long short-term memory network, obtains feature expression of a word, and uses a dependency relation to build a corresponding syntactic dependency tree for a clause in a paragraph; after combining the word feature vector and the syntactic dependency tree, adjacent dependency trees are connected by using a part of speech, a three-dimensional space-time graph is constructed, a relationship value in the graph is calculated by using an attention mechanism, convolution operation is performed on the generated space-time graph attention network, and an optimized word feature vector is output; and finally, after a mask operation, the aspect word is input into a softmax to obtain a sentiment polarity classification result. The application considers the timeliness of the review text, captures the fluctuation characteristics of the sentiment over time in the review, strengthens the processing capacity for long paragraphs and text with sentiment turning points, and makes the prediction effect of the classification more interpretable and more accurate.

Description

Technical Field

[0001] This invention relates to a text sentiment analysis method, and more particularly to an aspect-level sentiment analysis method based on a spatiotemporal graph attention network. Background Technology

[0002] Sentiment analysis, a task within the field of natural language processing, plays a crucial role in today's internet products. People typically express their opinions and emotions by posting comments online, and mining user opinions from these comments can drive the development of applications such as personalized recommendations and public services. Aspect-level sentiment analysis (ABSA) is a fine-grained task in sentiment classification, aiming to identify the sentiment polarity of target aspect words. Its characteristics can help users accurately pinpoint problems, thus attracting widespread attention in recent years. However, existing methods generally model a single paragraph uniformly, placing the entire comment text into a graph convolutional network, without considering the emotional fluctuations of the comment content over time. All words propagate feature information within a syntactic dependency tree, and most existing GCN-based methods employ a two-layer network structure, making it difficult for long-distance semantic information to be propagated to the features of the target aspect words through convolution. Since the relevant sentiment words of aspect words do not necessarily appear near the target word as in simple sentences, but may appear in the next sentence, this makes the model difficult to handle long comment paragraphs and does not achieve ideal prediction results when faced with comments with tone shifts. Summary of the Invention

[0003] Purpose of the invention: In order to solve the problems existing in the prior art, an aspect-level sentiment analysis method based on spatiotemporal graph attention network is proposed, which can improve the accuracy of sentiment analysis of each aspect in long comment paragraphs.

[0004] Technical Solution: The technical solution adopted in this invention is an aspect-level sentiment analysis method based on spatiotemporal graph attention networks, including:

[0005] The classification results of aspect word sentiment polarity of comment text are calculated using a heterogeneous spatiotemporal graph attention network model; the heterogeneous spatiotemporal graph attention network model includes:

[0006] The hidden vectors of aspect words and context in the comment text are calculated using a Long Short-Term Memory (LSTM) network, and the word feature vector corresponding to each word is output. Several clauses arranged in chronological order are used as the temporal dimension of the three-dimensional spatiotemporal graph, and the syntactic dependency trees corresponding to each clause are used as the spatial dimension of the three-dimensional spatiotemporal graph. The nodes in adjacent syntactic dependency trees are connected by part-of-speech relations, where part-of-speech relations refer to the node relations with the same part of speech. The directionality of part-of-speech relations is removed to obtain the three-dimensional spatiotemporal graph of the comment text.

[0007] The word feature vectors are assigned to nodes in the 3D spatiotemporal graph to obtain a 3D spatiotemporal graph with word feature vectors.

[0008] The three-dimensional spatiotemporal graph with word feature vectors is used to calculate the classification result of the sentiment polarity of aspect words in the comment text through a spatiotemporal graph attention network; the spatiotemporal graph attention network introduces an attention mechanism in the convolution process of both time and space dimensions.

[0009] The spatiotemporal graph attention network introduces an attention mechanism during the convolution process in both time and space dimensions. In the spatial dimension, attention propagation weights are introduced into the calculation of the spatial neighborhood information of the target word, and the calculation formula is as follows:

[0010]

[0011] in For the spatial neighborhood information of the target word, Let α represent the set of all neighboring words y that are directly connected to word x in the syntactic dependency tree. x，y h represents the attention propagation weights between the target word and its neighboring words in the spatial dimension. y This represents the feature vector of a word in a spatial neighborhood;

[0012] In calculating the temporal neighborhood information of the target word, attention propagation weights are also introduced, and the calculation formula is as follows:

[0013]

[0014] Where h temporal This represents the temporal neighborhood information of the target word. This indicates a direct connection to word x in adjacent syntactic dependency trees in the time dimension. t The neighboring word y t The set, h represents the attention propagation weights between the target word and its neighboring words in the time dimension. yt This represents the feature vector of words in the temporal neighborhood.

[0015] The calculation of attention propagation weights can be expressed in various forms; preferably, the calculation formula is as follows:

[0016]

[0017]

[0018] Where, α a For the standardized attention score of aspect words, β a For aspect-based attention scores, β i For contextual word attention scores, n sn is the number of sentences. i h is the number of words in the sentence. a h i These represent the target-related word feature expression and the context-related word feature expression, respectively. S represents the set of sentences in the paragraph, and P represents the paragraph.

[0019] Preferably, the long short-term memory network is a bidirectional long short-term memory network, which can enhance the semantic processing effect of the text after processing the text sequence in both forward and reverse directions, and obtain word feature vectors that combine contextual hidden information.

[0020] The stereo spatiotemporal graph with word feature vectors is used to calculate the classification result of the sentiment polarity of the aspect words of the comment text through a spatiotemporal graph attention network. The process includes: performing convolution operations in both time and space dimensions through the spatiotemporal graph attention network, and obtaining the enriched word feature vectors after multiple convolutions.

[0021] The formula for calculating convolution in the spatial dimension is:

[0022]

[0023] in To represent the spatial features of the target word in the spatiotemporal graph attention network convolutional layer l, For the spatial neighborhood information of the target word, denoted as the feature representation vector of the target word in the previous layer, W as the transformation weight, b as the bias term, and σ as the non-linear activation function of this layer.

[0024] After obtaining the spatial feature representation of the target word, a convolution operation is performed in the temporal dimension. The calculation formula is as follows:

[0025]

[0026] in, For the temporal neighborhood information of the target word, W t and b t These are the transformation weights and bias terms used in the time dimension calculation, respectively. This is the final feature representation of a word after it has been aggregated through a spatiotemporal graph attention network, i.e., the enriched word feature vector.

[0027] The enriched word feature vectors are masked to filter out aspect word feature vectors. These vectors are then passed through the output layer of a spatiotemporal graph attention network to obtain the classification result of the sentiment polarity of the aspect words in the comment text. Preferably, the output layer of the spatiotemporal graph attention network uses a softmax function.

[0028] The heterogeneous spatiotemporal graph attention network model is trained using a comment text dataset, and the loss function used in the training process is:

[0029]

[0030] Where N represents the number of training samples, C represents the number of classifications, and y p Indicates true emotional tags, To predict the label, three emotional outcomes are given: "positive", "neutral", and "negative". λ is the regularization coefficient, Θ is the model training parameter, and ||2 is the second paradigm calculation.

[0031] This invention provides an aspect-level sentiment analysis system for aspect-level sentiment analysis of comment text, including a heterogeneous spatiotemporal graph attention network model, wherein the heterogeneous spatiotemporal graph attention network model includes:

[0032] Long Short-Term Memory (LSTM) network is used to calculate the hidden vectors of aspect words and context in the comment text, output the word feature vector corresponding to each word and send it to the data processing module;

[0033] The data processing module is used to: use several clauses of the comment text arranged in chronological order as the time dimension of a three-dimensional spatiotemporal graph; use the syntactic dependency tree corresponding to each clause as the spatial dimension of the three-dimensional spatiotemporal graph; connect nodes in adjacent syntactic dependency trees through part-of-speech relations, where part-of-speech relations refer to node relations with the same part of speech; remove the directionality of part-of-speech relations to obtain a three-dimensional spatiotemporal graph of the comment text; assign the word feature vectors to the nodes in the three-dimensional spatiotemporal graph to obtain a three-dimensional spatiotemporal graph with word feature vectors and send it to the spatiotemporal graph attention network;

[0034] The attention mechanism module is used to calculate the attention propagation weights and send them to the spatiotemporal graph attention network;

[0035] The spatiotemporal graph attention network is used to calculate the classification result of the sentiment polarity of aspect words in the comment text based on the three-dimensional spatiotemporal graph with word feature vectors; the spatiotemporal graph attention network introduces an attention mechanism in the convolution process of both time and space dimensions.

[0036] The present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described aspect-level sentiment analysis method based on a spatiotemporal graph attention network.

[0037] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described aspect-level sentiment analysis method based on a spatiotemporal graph attention network.

[0038] The beneficial effects achieved by the present invention using the above scheme are as follows: (1) The present invention constructs a heterogeneous three-dimensional model, a spatiotemporal graph convolutional network for text data, which can capture the features of the target object changing over time, and thus incorporate the feature expression of the time dimension into the object. Specifically, the comment text is processed separately based on time and semantics, and multiple syntactic dependency trees are built. Then, adjacent dependency trees are connected to capture the time dimension features of the comment for the aspect words, and at the same time, the feature information of emotional fluctuation is incorporated into the feature expression of the aspect words. Through the above method, the present invention achieves more accurate prediction of the emotional polarity of aspect words. The prediction and classification effect is better than other comparative models in the experimental results on the ACL-14Twitter, SemEval 2014, SemEval2015 and SemEval 2016 datasets. (2) The present invention proposes to extend the node connection in the time dimension from one-to-one to many-to-many connection method, which solves the problem that it is not suitable for text data. (3) This invention proposes to incorporate an attention mechanism into a spatiotemporal graph convolutional network to calculate the weights of propagation relationships, thereby constructing a spatiotemporal graph attention network. This enhances the model's ability to capture contextual sentiment information for aspect words. This invention achieves the best results of 89.03 and 80.86 in Accuracy and F1 scores, respectively, with an accuracy improvement of 1.6% to 14.5% compared to the control method. Attached Figure Description

[0039] Figure 1 This is a flowchart of an aspect-level sentiment analysis method based on a spatiotemporal graph attention network according to the present invention;

[0040] Figure 2 A schematic diagram illustrating the idea of ​​transforming a syntactic dependency tree into a spacetime graph structure;

[0041] Figure 3 This is a structural diagram of the heterogeneous spatiotemporal graph attention network framework H-STGAN (Heterogeneous SpatioTemporal Graph Attention Network) proposed in this invention;

[0042] Figure 4 This is a comparison chart of the experimental results of the H-STGAN model of this invention under different numbers of convolutional layers. Detailed Implementation

[0043] To make the technical solutions and advantages of the present invention clearer, the specific technical solutions of the present invention will be described in detail below with reference to the accompanying drawings and embodiments. The present invention is not limited to the scope of the specific embodiments.

[0044] like Figure 1The diagram shows the basic flow of the aspect-level sentiment analysis method based on spatiotemporal graph attention networks described in this invention. This flow is further explained below in five parts, S1 to S5. Parts S1 and S2 can be performed simultaneously, without any order restriction.

[0045] S1, Given a comment text, a Long Short-Term Memory (LSTM) network is used to learn the hidden vectors of aspect words and context in the comment text, and output the feature vector corresponding to each word. In this embodiment, a bidirectional LSM network is preferred.

[0046] First, define the comment text. Given a paragraph P = {S1, S2, ..., S...} containing n clauses. n}, where the sentence S = {w1, w2, ..., ω} τ+1 ,…,ω τ+m ,…,w n}, within the sentence, includes a set of aspect words of length m, a = {ω} τ+1 ,…,ω τ+m To enable the model to process natural language, sentences need to be embedded and represented as vectors. This method uses the GloVe word embedding model, which combines word co-occurrence to obtain vectors, enabling better statistical analysis of global information. The sentence is then processed through the embedding matrix M∈R... d×V Then word vectors are obtained. Where e∈R d d represents the dimension of the embedding, and V is the vocabulary size;

[0047] After obtaining the vector representation of the words, such as Figure 3 As shown in the lower left part, when input into the Bi-LSTM model, after passing through its forward and backward processing sequences, the expression of each word combined with contextual hidden information can be obtained. This process is shown in the formula:

[0048]

[0049]

[0050] The upper formula represents the forward representation learning process, and the lower formula represents the backward process. θ represents the LSTM model parameters. Bi-LSTM concatenates the bidirectional representation vectors, and the final output is represented as... Where i∈[1,n]; the hidden layer representation of the final sentence is H. S ={h1, h2, ..., h τ+1 , ..., h τ+m , ..., h n}∈R n×2d .

[0051] S2, for the comment text in S1, uses several clauses arranged in chronological order as the time dimension of the three-dimensional spatiotemporal graph, and the dependency tree corresponding to each clause as the spatial dimension of the three-dimensional spatiotemporal graph; in the planar space, the dependency relations are used to build all the words in each clause into the corresponding syntactic dependency tree, and in the three-dimensional space, the nodes in adjacent dependency trees are connected by part-of-speech relations, that is, nodes with the same part of speech are connected, and the directionality of part-of-speech relations is removed to construct the three-dimensional spatiotemporal graph.

[0052] Syntactic dependency trees can provide structure to natural language. Their construction tools analyze the grammatical structure of the input sentence and use dependency relations to build a syntactic dependency tree. To utilize spatiotemporal graph neural networks, the comment text needs to be divided into clauses according to time and semantics. Each clause is then used to build a corresponding syntactic dependency tree, thus obtaining graph structure data. Adjacent dependency trees are connected through part-of-speech tags, removing the directionality of the relationships, and constructing a three-dimensional spatiotemporal graph. This process is as follows: Figure 2 As shown;

[0053] This method will use the spaCy tool to construct the syntactic dependency tree, such as Figure 3 As shown in the lower right part, the nodes in this tree are directed graphs with part-of-speech tags. After representing the sentences in paragraph P as dependency trees, they are connected sequentially. The part-of-speech of words is used as the connection basis. For example, all noun nodes in one tree are connected to all nouns in the next time dependency tree, with the connection relationship being n:m, where n≥0 and m≥0. After removing the directivity from the syntactic dependency tree, the entire paragraph is constructed as a three-dimensional undirected graph, and finally, the corresponding adjacency matrix is ​​obtained. Where n P The total number of words in a paragraph is calculated using the formula shown below:

[0054]

[0055] S3 combines the word feature vectors obtained from S1 and S2 with the 3D spatiotemporal graph. Word vectors are then filled into the nodes of the 3D spatiotemporal graph, resulting in a 3D spatiotemporal graph with word feature vectors. This graph serves as the input data for the spatiotemporal graph attention network. An attention mechanism is then used to calculate weight coefficients for all connections in this 3D spatiotemporal graph. Here, "connections" refer to spatial dependencies and temporal part-of-speech relationships.

[0056] Attention mechanisms can calculate the importance of relationships to the target object and control the propagation distance between each word by calculating the relationship weights in the network, so that context words that have different effects on the other side can play their respective roles.

[0057] After obtaining the feature vectors of the words and the 3D spatiotemporal graph, the two are combined. Word vectors are filled in by assigning them to nodes in the graph. Then, an attention mechanism is used to calculate weight coefficients for the relationships in the graph, ultimately generating a spatiotemporal graph attention network. The formula for calculating the attention score in the model is as follows:

[0058]

[0059] Where n s n is the number of sentences. i h is the number of words in the sentence. a and h i The target word feature representation and the context word feature representation are respectively used. The total score obtained is standardized to obtain the attention weight of the relationship in the model. The calculation formula is as follows:

[0060]

[0061] When a convolutional network performs neighborhood aggregation, the weights will filter other word expressions within the neighborhood to varying degrees based on the needs of the aspect words.

[0062] S4 involves performing convolution operations on the three-dimensional spatiotemporal graph with word feature vectors through a spatiotemporal graph attention network in both time and space dimensions, resulting in enriched word feature vectors after multiple convolutions.

[0063] Graph Convolutional Networks (GCNs) utilize neighborhood aggregation to implement convolution operations in non-Euclidean data. Based on this idea, this method uses an undirected dependency tree built from clauses corresponding to a time as a planar spatial graph. By superimposing multiple time periods, a heterogeneous three-dimensional model of spatiotemporal graph convolutional network for text data is constructed. This model can capture the features of the target object that change over time, thereby incorporating the time dimension into the feature expression of the object.

[0064] This method constructs a planar spatial graph using dependency relationships, then overlays each layer of the spatial graph, using time (t) and semantics as the basis for layering. Part-of-speech tags are used to connect words in adjacent spatial graphs. The temporal connections here are multi-type in content and exhibit a many-to-many pattern in connection structure. This method defines this heterogeneous spatiotemporal graph as... in Let (x, r, y) be a set of parts of speech and dependency relations, and (x, r, y) be a triple that describes the fact that the entity word starting with x is connected to the entity word ending with y through relation r.

[0065] The specific calculation process of convolution operation in heterogeneous spatiotemporal graphs is represented by the following formula:

[0066]

[0067] in Represented as a word adjacency matrix incorporating self-connections. is the degree matrix of the adjacency matrix, used to control the contribution of the number of neighboring nodes to the convolution operation, and h and w are the word feature representation and training weights, respectively;

[0068] like Figure 3 As shown in the middle section, in the spatiotemporal graph convolution module, this method first performs convolution on the planar space, and then uses a syntactic dependency tree to capture the contextual semantic information within the sentence for aspect words. Here, we define... Let represent the set of all neighboring words y that are directly connected to word x in a single-layer plane, i.e., within a dependency tree. The calculation process for planar spatial convolution is as follows:

[0069]

[0070] in For the spatial neighborhood information of the target word, α x，y To determine the attention propagation weights between the target word and its neighboring words, after obtaining the neighborhood representation, we calculate the spatial dimension feature representation:

[0071]

[0072] in Let W be the spatial feature representation of the word convolved into the l-th layer of graph G, where W is the transformation weight, b is the bias term, and σ represents the non-linear activation function of the layer.

[0073] To learn temporal features, the model needs to perform stereo temporal convolution, which is defined here. This indicates that in the time dimension, i.e., in adjacent syntactic dependency trees, it is directly connected to word x. t The neighboring word y t The set, the stereo temporal convolution calculation process is as follows:

[0074]

[0075] Where h temporal Representing the temporal neighborhood information of the target word, after obtaining the spatial dimension feature representation and the temporal dimension neighborhood representation, the expression formula for the next layer of the spatiotemporal graph neural network is calculated as follows:

[0076]

[0077] in This is the final feature representation of a word after it has been aggregated through a spatiotemporal graph attention network. At this point, the feature has captured the semantic information of the sentence in the spatial dimension and obtained the emotional fluctuation features between sentences over time in the temporal dimension.

[0078] S5 performs a masking operation on all word feature vectors to filter out the feature vectors of aspect words, inputs them into the softmax layer to obtain the prediction probability, then uses the loss function to optimize the training parameters, performs multiple model trainings, and finally uses the model to predict the classification result of the sentiment polarity of aspect words.

[0079] After learning through the spatiotemporal graph attention network, the feature representations of all words in paragraph P are obtained. This method's model needs to process the hidden vectors of non-aspect words in the comment text through a masking operation to filter out the feature representations of aspect words. Here, 0 is used to replace the vectors of non-aspect words. The masking operation retains only the feature representations of aspect words, ultimately resulting in the sentence representation H. mask ={0, ..., h s·t(τ+1) , ..., h st(τ+m) …, 0}, in the paragraph

[0080] The final feature vector is input into a linear layer, and the sentiment polarity of the aspect words is obtained through softmax, as shown in the formula:

[0081] p = softmax(W o o+b o )

[0082] Where p is the predicted sentiment polarity distribution, W o and b o These are the training weights and biases of the output layer;

[0083] After obtaining the prediction results, the loss function is used to train the parameters and optimize the model. The loss function of the model is defined as follows:

[0084]

[0085] Where N represents the number of training samples, C represents the number of classifications, and y p Indicates true emotional tags, To predict the label, three emotional outcomes are given: "positive", "neutral", and "negative". λ is the regularization coefficient, Θ is the model training parameter, and ||2 is the second paradigm calculation.

[0086] Experimental dataset:

[0087] The five datasets used in the experiments of this invention are the ACL-14 Twitter comment dataset, the Lap14 and Rest14 datasets in SemEval2014, the Rest15 dataset in SemEval 2015, and the Rest16 dataset in SemEval 2016. Each data point in these datasets corresponds to three categories of content: comment text, aspect words, and sentiment polarity, which are suitable for the model of this invention. The polarity of the five datasets and the number of training / test splits are shown in Table 1.

[0088] Table 1 Experimental Dataset Information

[0089]

[0090] Model parameter settings:

[0091] In the experiments, the word embedding dimension was set to 300 for the GloVe pre-trained model, and the hidden layer dimension was set to 300 for the Bi-LSTM bidirectional memory network, with the L2 regularization coefficient set to 10. -5 Using a learning rate of 10 -3 Adam was used as the optimizer to optimize the model, and the batch sample size was finally set to 32.

[0092] To verify the effectiveness of the H-STGAN model, we will compare it with the following models:

[0093] SVM: Supervised learning using support vector machines to complete sentiment classification tasks;

[0094] LSTM: The hidden layer output of the sentence is obtained through a Long Short-Term Memory network, and the sentiment polarity is output by a classifier.

[0095] IAN: It uses interactive attention mechanism to learn the relationship between aspect words and context words and changes the weight coefficients.

[0096] MGAN proposes a multi-granularity attention mechanism that combines coarse-grained and fine-grained attention to learn the relationship between aspect words and context words;

[0097] ASGCN proposes to construct sentences as graph structures using syntactic dependency trees and apply GCN to aspect-level sentiment analysis tasks. It learns feature information of aspect words and syntactic structures through graph structure networks.

[0098] BiGCN: Constructs a two-layer interactive graph convolutional network structure, which integrates word co-occurrence information and dependency type information using a hierarchical graph structure to learn hierarchical information;

[0099] DGEDT: An aspect-level sentiment analysis model that combines GCN and bidirectional Transformer structure, reducing noise generated by erroneous dependency trees;

[0100] DualGCN: Proposes a dual graph convolutional network model that can simultaneously consider the complementarity of grammatical structures and semantic relevance, thus alleviating the problem of irregular sentence structure.

[0101] Analysis of experimental results:

[0102] This invention uses Accuracy and F1 macro average as evaluation indicators for the experiment. The experimental results are shown in Table 2, where the bolded indicators are the optimal results.

[0103] Table 2 Experimental Results

[0104]

[0105] Experiments show that H-STGAN outperforms the control model across five datasets, validating the effectiveness of the proposed model. The following analysis uses the mean of all evaluation metrics as a basis. In the control models, IAN and MGAN utilize attention mechanisms to learn semantic information. By adjusting the weights of aspect words and context words, they achieve a 1.4%-5.7% improvement in accuracy compared to basic models like SVM and LSTM. Furthermore, a well-designed attention mechanism can improve prediction performance by approximately 2%. ASGCN, utilizing a graph convolutional network, achieves approximately a 1.3% accuracy improvement over the baseline model using attention mechanisms, validating that irregular graph structures better leverage the semantic information in text data. By mining grammatical structures and propagating word vector information, aspect word features are better expressed. BiGCN and DualGCN build dual-channel graph convolutional networks to learn hidden features, addressing the issues of informal sentence structures and content expression. Compared to single-channel models, they achieve higher accuracy. The GCN model has shown considerable improvement. BiGCN, DGEDT, and DualGCN models, through reasonable application of GCN and addressing the issue of non-standard syntactic dependency trees, achieve a 2%-4.1% improvement in prediction accuracy compared to ASGCN. H-STGAN, utilizing an attention mechanism and modeling spatiotemporal graph convolutions on comment text, achieves a 1.6%-14.5% improvement in accuracy compared to the control model. This verifies that spatiotemporal graph convolutional networks can propagate temporal features, extracting the emotional fluctuations between different clauses within a comment and providing better sentiment polarity judgment for aspect words. On the Twitter dataset, due to the short comment texts, H-STGAN's accuracy improvement compared to DualGCN is small, and its F1 score is lower than DualGCN. However, on the Rest14 dataset, which contains many long texts, H-STGAN performs better than on other datasets, demonstrating that the model of this invention is suitable for processing complex or long comment text data.

[0106] Model structure analysis:

[0107] This section will analyze the model and discuss the performance of the H-STGAN model under different numbers of spatiotemporal graph convolutional network layers. Experiments were conducted on five datasets with the spatiotemporal graph convolutional layers set to 1, 2, 3, 4, 5 and 6 layers respectively, while keeping other aspects of the model structure unchanged.

[0108] like Figure 4As shown, when the number of layers in the spatiotemporal graph convolutional network increases from 1 to 2, both the accuracy and F1 score improve. However, when the number of layers increases from 2 to 3, the F1 score decreases except on the Lap14 dataset, showing an overall downward trend. Furthermore, as the number of layers in the spatiotemporal graph convolutional network increases from 3 to 6, the performance of the H-STGAN model deteriorates across all datasets. In summary, the model performs best with 2 spatiotemporal graph convolutional layers. When the number of layers exceeds 2, nodes in the network over-propagate, causing the captured neighborhood features to be mixed with a large amount of noise, leading to a loss of model effectiveness.

[0109] Ablation experiment:

[0110] Table 3 Results of the ablation experiment

[0111]

[0112]

[0113] Table 3 shows the results of ablation experiments after removing different modules from H-STGAN. Here, w / o attention means removing the attention calculation mechanism and using the original adjacency matrix as the propagation weights; w / o GCN-spatio means removing the spatial convolution operation in H-STGAN, that is, no longer performing feature convolution within a single clause; w / o GCN-temporal means removing the temporal convolution operation in H-STGAN, that is, no longer performing feature convolution between adjacent clauses; w / o Bi-LSTM means eliminating the hidden vector learning process of the bidirectional long short-term memory network. Experimental results show that spatial convolution operations, i.e., the original GCN convolution operations, contribute the most to the model, and removing them reduces accuracy by up to 12.48%. Removing the attention mechanism and the Bi-LSTM learning process causes the model's performance to decline on all datasets, demonstrating the role of the attention mechanism and Bi-LSTM in learning the relationship between words and context words. Removing the temporal convolution operation reduces the model's prediction performance by about 3.5%, which is close to the performance of traditional GCN aspect-level sentiment analysis models, proving that temporal graph convolution plays a good role in improving the model's prediction performance.

[0114] In one embodiment, an aspect-level sentiment analysis system is provided for aspect-level sentiment analysis of comment text, including a heterogeneous spatiotemporal graph attention network model, wherein the heterogeneous spatiotemporal graph attention network model includes:

[0115] Long Short-Term Memory (LSTM) network is used to calculate the hidden vectors of aspect words and context in the comment text, output the word feature vector corresponding to each word and send it to the data processing module;

[0116] The data processing module is used to: use several clauses of the comment text arranged in chronological order as the time dimension of a three-dimensional spatiotemporal graph; use the syntactic dependency tree corresponding to each clause as the spatial dimension of the three-dimensional spatiotemporal graph; connect nodes in adjacent syntactic dependency trees through part-of-speech relations, where part-of-speech relations refer to node relations with the same part of speech; remove the directionality of part-of-speech relations to obtain a three-dimensional spatiotemporal graph of the comment text; assign the word feature vectors to the nodes in the three-dimensional spatiotemporal graph to obtain a three-dimensional spatiotemporal graph with word feature vectors and send it to the spatiotemporal graph attention network;

[0117] The attention mechanism module is used to calculate the attention propagation weights and send them to the spatiotemporal graph attention network;

[0118] The spatiotemporal graph attention network is used to calculate the classification result of the sentiment polarity of aspect words in the comment text based on the three-dimensional spatiotemporal graph with word feature vectors; the spatiotemporal graph attention network introduces an attention mechanism in the convolution process of both time and space dimensions.

[0119] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described aspect-level sentiment analysis method based on a spatiotemporal graph attention network.

[0120] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the aforementioned aspect-level sentiment analysis method based on a spatiotemporal graph attention network.

[0121] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0122] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0123] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

Claims

1. A method for aspect-level sentiment analysis based on a spatio-temporal graph attention network, characterized in that, The application relates to a method for calculating the aspect word sentiment polarity of a review text. The heterogeneous spatio-temporal graph attention network model comprises: The hidden vectors of the aspect words and the context in the review text are calculated by a long short-term memory network, and a word feature vector corresponding to each word is output; a plurality of clauses arranged in a time sequence are taken as the time dimension of a three-dimensional spatio-temporal graph, the syntactic dependency trees corresponding to the clauses are taken as the space dimension of the three-dimensional spatio-temporal graph, and the nodes in adjacent syntactic dependency trees are connected through a part-of-speech relationship, wherein the part-of-speech relationship refers to a node relationship with the same part of speech, and the directionality of the part-of-speech relationship is removed, so that the three-dimensional spatio-temporal graph of the review text is obtained; The word feature vector is assigned to the nodes in the three-dimensional spatio-temporal graph, so that the three-dimensional spatio-temporal graph with the word feature vector is obtained; The three-dimensional spatio-temporal graph with the word feature vector is calculated by the heterogeneous spatio-temporal graph attention network, so that the classification result of the aspect word sentiment polarity of the review text is obtained; the attention mechanism is introduced in the convolution process in the time and space dimensions of the heterogeneous spatio-temporal graph attention network; The three-dimensional spatio-temporal graph with the word feature vector is calculated by the spatio-temporal graph attention network, so that the classification result of the aspect word sentiment polarity of the review text is obtained, which comprises: the convolution operation in the time and space dimensions is performed by the spatio-temporal graph attention network, and the word feature vector is enriched after multiple convolutions; The convolution calculation formula in the space dimension is: After the spatial feature expression of the target word is obtained, the convolution operation in the time dimension is performed, and the calculation formula is: ， wherein is a spatial feature representation of the target word at the i-th layer of the spatio-temporal graph attention network, is a spatial feature representation of the target word at the i-th layer of the spatio-temporal graph attention network, is spatial neighborhood information of the target word, is a feature representation vector of the target word at the i-th layer of the spatio-temporal graph attention network, is a transformation weight, is a bias term, denotes a non-linear activation function of the layer. The enriched word feature vector is subjected to a mask operation, and the feature vector of the aspect word is filtered, so that the classification result of the aspect word sentiment polarity of the review text is obtained through the output layer of the spatio-temporal graph attention network. ， wherein, is the time neighborhood information of the target word, and are the transformation weight and bias term respectively in time dimension calculation, is the final feature expression of the word after the space-time graph attention network is aggregated once, i.e., the enriched word feature vector; The attention mechanism is introduced in the convolution process in the time and space dimensions of the spatio-temporal graph attention network, and the attention propagation weight is introduced in the calculation of the spatial neighborhood information of the target word, and the calculation formula is:

2. The aspect-level sentiment analysis method based on the spatio-temporal graph attention network according to claim 1, characterized in that, The attention propagation weight is also introduced in the calculation of the time neighborhood information of the target word, and the calculation formula is: ， in For the spatial neighborhood information of the target word, This represents all direct links to words within the syntactic dependency tree. Neighborhood words The set, The attention propagation weights between the target word and its neighboring words in the spatial dimension. Feature vectors representing spatial neighborhood words; The calculation formula of the attention propagation weight is: ， wherein denotes the time neighborhood information of the target word, denotes a set of neighborhood words directly connected to the word in the syntactic dependency tree adjacent in the time dimension, is the attention propagation weight between the target word and the neighborhood word in the time dimension, denotes the feature vector of the time neighborhood word.​ 3. The aspect-level sentiment analysis method based on the spatio-temporal graph attention network according to claim 2, characterized in that, The long short-term memory network adopts a bidirectional long short-term memory network. ， ， wherein, is a standardized attention score for an aspect word, is an aspect word attention score, is a context word attention score, is a number of sentences, is a number of words within a sentence, , are a target aspect word feature representation and a context word feature representation, respectively, S denotes a set of sentences in a passage, and P denotes a passage.

4. The aspect-level sentiment analysis method based on the spatio-temporal graph attention network according to claim 1, characterized in that, The output layer of the spatio-temporal graph attention network adopts a softmax function.

5. The aspect-level sentiment analysis method based on the spatio-temporal graph attention network according to claim 1, characterized in that, The heterogeneous spatio-temporal graph attention network model is trained through a review text data set, and the loss function adopted in the training process is:

6. The method of aspect-level sentiment analysis based on spatio-temporal graph attention network according to claim 1, characterized in that, The application relates to a method for calculating the aspect word sentiment polarity of a review text. ， wherein denotes the number of training samples, denotes the number of classes, denotes the true sentiment label, is the predicted label, classified into three emotional outcomes of "positive", "neutral", "negative", is the regularization coefficient, is the model training parameter, is the second norm calculation.

7. An aspect-level sentiment analysis system for implementing the aspect-level sentiment analysis method based on the spatio-temporal graph attention network according to any one of claims 1-6, characterized in that: The application relates to a method for calculating the aspect word sentiment polarity of a review text. The long short-term memory network is used for calculating the hidden vectors of the aspect words and the context in the review text, and outputting the word feature vector corresponding to each word and sending the word feature vector to the data processing module. The data processing module is configured to: arrange a plurality of clauses of the review text in time sequence as a time dimension of a three-dimensional space-time graph, arrange a syntax dependency tree corresponding to each clause as a space dimension of the three-dimensional space-time graph, and connect nodes in adjacent syntax dependency trees through a part-of-speech relationship, wherein the part-of-speech relationship refers to a node relationship with the same part of speech, remove the directionality of the part-of-speech relationship, and obtain the three-dimensional space-time graph of the review text; and assign the word feature vector to the nodes in the three-dimensional space-time graph, obtain the three-dimensional space-time graph with the word feature vector, and send the three-dimensional space-time graph with the word feature vector to the space-time graph attention network. The attention mechanism module is configured to calculate attention propagation weights and send the attention propagation weights to the space-time graph attention network. The space-time graph attention network is configured to calculate a classification result of aspect word sentiment polarity of the review text according to the three-dimensional space-time graph with the word feature vector, and introduce an attention mechanism in a convolution process in the time and space dimensions.

8. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the aspect-level sentiment analysis method based on the space-time graph attention network in any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the aspect-level sentiment analysis method based on the space-time graph attention network in any one of claims 1 to 6.

Images