Implicit sentiment analysis method, system and equipment based on iterative graph structure learning and medium

Abstract

The invention discloses an implicit sentiment analysis method, system and device based on iterative graph structure learning and a medium, and relates to the technical field of implicit sentiment analysis of natural language processing. The method comprises the steps that an obtained target text is input into a KIG model for original graph structure optimization fusion, and an implicit graph structure is obtained; the implicit graph structure is used for expressing implicit sentiment analysis; the KIG model comprises a text encoder, a graph construction layer, a graph learning layer and a graph fusion layer; wherein the text encoder is used for performing high-dimensional space mapping on an implicit emotion text by utilizing a dictionary to obtain a word vector; the graph construction layer is used for constructing a plurality of initial graph structures in different view forms based on the word vectors; the graph learning layer is used for carrying out feature extraction on each initial graph structure in an iteration mode; and the image fusion layer is used for integrating the features of each view. According to the method, different view information can be fused, a graph structure learning framework is iteratively learned, and the flexibility of the model in an implicit sentiment analysis task is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of implicit sentiment analysis in natural language processing, and in particular to an implicit sentiment analysis method, system, device and medium based on iterative graph structure learning. Background Art

[0002] In recent years, graph neural networks have achieved good results in implicit sentiment analysis, but there are still some problems. On the one hand, the implicit sentiment text of the single-view graph structure lacks structural information that can accurately capture the fuzzy emotional expression. For a single view, each node or entity in the data set can only represent the graph structure from one perspective or source, that is, each entity has only one representation, which is different from the multi-view graph structure containing multiple source information. The single-view graph structure cannot provide different aspects or attributes of the entity. On the other hand, the preset fixed graph structure may contain some edge noise, resulting in the inability to use an accurate topological structure to represent sentiment information. In the preset fixed graph structure, the structure of the graph is pre-defined and fixed. The number of nodes and the relationship between nodes in this structure are fixed and will not change with the processing of data. However, in machine learning and deep learning, this fixed graph structure requires a lot of manpower and uses prior knowledge to guide the learning process of the algorithm, especially when processing graph structure data, such as graph embedding, graph convolutional network, etc. In practical applications, the structure of the graph often changes dynamically, so a more flexible way is needed to handle the evolution of graph structure data. Summary of the invention

[0003] The purpose of the present invention is to provide an implicit sentiment analysis method, system, device and medium based on iterative graph structure learning, which can integrate different view information and iteratively learn the graph structure learning framework to improve the flexibility of the model in implicit sentiment analysis tasks.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] An implicit sentiment analysis method based on iterative graph structure learning, comprising:

[0006] Get the original graph structure of the target text;

[0007] The original graph structure is input into the KIG model for optimization and fusion to obtain an implicit graph structure; the implicit graph structure is used to express implicit sentiment analysis; the KIG model includes a text encoder, a graph construction layer, a graph learning layer, and a graph fusion layer;

[0008] Among them, the text encoder is used to use a dictionary to perform high-dimensional space mapping on implicit emotional text to obtain word vectors; the graph construction layer is used to construct multiple initial graph structures in different view forms based on the word vectors; the graph learning layer is used to extract features from each of the initial graph structures in an iterative manner; and the graph fusion layer is used to integrate the features of each view.

[0009] Optionally, the views include views of co-occurrence statistics, cosine similarity and syntactic dependency tree construction.

[0010] Optionally, the operation expression of the co-occurrence statistics is:

[0011] PPMI(w i ,w j )=max{PMI(w i ,w j ),0}

[0012] Among them, PPMI represents positive point mutual information, which is a reweighted form of co-occurrence metric; w i represents the i-th word; w j represents the jth word; PMI represents point mutual information.

[0013] Optionally, the operation expression of the cosine similarity is:

[0014]

[0015] Among them, Cosine Similarity represents cosine similarity; A and B represent w i and w j word embedding representation.

[0016] Optionally, the operation expression of the syntactic dependency tree is:

[0017]

[0018] Among them, A ij The matrix representing the syntactic dependency structure; DT(w i ,w j ) indicates w i and w j Relationships in syntactic structures.

[0019] Optionally, the operation expression of the graph learning layer is:

[0020]

[0021] in, is the vector representation of the t-th iteration under the i-th view of the implicit sentiment part, using the Relu activation function, GCN represents graph convolution, X represents the original node feature matrix, Represents the optimized graph after combining the learning graph with the original graph.

[0022] Optionally, the operation expression of the graph fusion layer is:

[0023]

[0024] in, Refers to the value logit of pronoun j at viewpoint 1 after t rounds of iterations.

[0025] The present invention also provides an implicit sentiment analysis system based on iterative graph structure learning, comprising:

[0026] Data acquisition module, used to obtain the original graph structure of the target text;

[0027] An optimization and fusion module is used to input the original graph structure into the KIG model for optimization and fusion to obtain an implicit graph structure; the implicit graph structure is used to express implicit sentiment analysis; the KIG model includes a text encoder, a graph construction layer, a graph learning layer, and a graph fusion layer;

[0028] Among them, the text encoder is used to use a dictionary to perform high-dimensional space mapping on implicit emotional text to obtain word vectors; the graph construction layer is used to construct multiple initial graph structures in different view forms based on the word vectors; the graph learning layer is used to extract features from each of the initial graph structures in an iterative manner; and the graph fusion layer is used to integrate the features of each view.

[0029] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the implicit sentiment analysis method based on iterative graph structure learning.

[0030] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the implicit sentiment analysis method based on iterative graph structure learning as described above.

[0031] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0032] The present invention discloses an implicit sentiment analysis method, system, device and medium based on iterative graph structure learning, the method comprising: inputting the acquired target text into a KIG model to optimize and fuse the original graph structure to obtain an implicit graph structure; the implicit graph structure is used to express implicit sentiment analysis; the KIG model comprises a text encoder, a graph construction layer, a graph learning layer and a graph fusion layer; wherein the text encoder is used to map the implicit sentiment text to a high-dimensional space using a dictionary to obtain a word vector; the graph construction layer is used to construct a plurality of initial graph structures in different view forms based on the word vector; the graph learning layer is used to extract features from each initial graph structure in an iterative manner; and the graph fusion layer is used to integrate the features of each view. The present invention is capable of fusing different view information and iteratively learning a graph structure learning framework, thereby improving the flexibility of the model in implicit sentiment analysis tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0034] Figure 1 It is a flow chart of the implicit sentiment analysis method based on iterative graph structure learning of the present invention;

[0035] Figure 2 This is a KIG model framework diagram in this embodiment. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] The purpose of the present invention is to provide an implicit sentiment analysis method, system, device and medium based on iterative graph structure learning, which can integrate different view information and iteratively learn the graph structure learning framework to improve the flexibility of the model in implicit sentiment analysis tasks.

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] like Figure 1As shown, the present invention provides an implicit sentiment analysis method based on iterative graph structure learning, comprising:

[0040] Step 100: Obtain the original graph structure of the target text.

[0041] Step 200: Input the original graph structure into the KIG model for optimization and fusion to obtain an implicit graph structure; the implicit graph structure is used to express implicit sentiment analysis; the KIG model includes a text encoder, a graph construction layer, a graph learning layer, and a graph fusion layer.

[0042] Among them, the text encoder is used to use a dictionary to perform high-dimensional space mapping on implicit emotional text to obtain word vectors; the graph construction layer is used to construct multiple initial graph structures in different view forms based on the word vectors; the graph learning layer is used to extract features from each of the initial graph structures in an iterative manner; and the graph fusion layer is used to integrate the features of each view.

[0043] Based on the above technical solution, the following embodiments are provided.

[0044] In this embodiment, the iterative graph structure learning framework (KIG) uses the rich multi-source information provided by prior knowledge and performs implicit sentiment analysis based on three views: co-occurrence statistics, cosine similarity, and syntactic dependency tree, which helps the model capture implicit and ambiguous emotional expressions. In addition, KIG iterates the three original graph structures and extracts their implicit graph structures to better fit the data itself and optimize the downstream implicit sentiment analysis tasks.

[0045] The KIG model is divided into four modules, namely text encoder, graph construction, graph learning, and graph fusion. First, the text encoder maps the implicit sentiment text to a high-dimensional space through a dictionary to obtain a word embedding matrix. The text is also converted into a word vector and used as the input of the model. Secondly, the graph construction layer constructs the initial graph structure through three different views to provide multi-source information for downstream feature extraction tasks. After that, the graph learning layer performs feature extraction and graph structure learning in an iterative manner. The learned node embeddings can provide useful information for the model to learn a better graph structure, thereby finding a suitable graph structure for the implicit sentiment analysis task to strengthen the original graph structure. Finally, the graph fusion layer integrates the features of each view and finds a consistent decision boundary.

[0046] The KIG model framework is as follows Figure 2 As shown in the figure, w i represents the i-th word, l represents the number of layers in the graph convolutional network (GCN), Refers to the hidden layer vector representation of the graph structure after t rounds of iterative optimization of the i-th viewpoint, Refers to the value logit of pronoun j at viewpoint 1 after t rounds of iterations.

[0047] Text encoding layer:

[0048] In this layer, each word in the text will be encoded into D = {w1,w2,...,w n}, where w i is the word to be encoded. This embodiment maps these words to a high-dimensional vector space to obtain a word embedding matrix X, where X is in the real number field n×d, and n and d represent the filling length and dimension of the vector, respectively. After that, this embodiment can construct a graph structure of the text through the following three different views.

[0049] Graph construction layer:

[0050] (1) Co-occurrence statistics

[0051] Word co-occurrence statistical analysis methods are widely used in text mining research. Positive point-wise mutual information (PPMI) is a reweighted form of co-occurrence measurement and is considered to be the most advanced model for measuring the similarity between two words. PPMI is a variant of point-wise mutual information (PMI), which measures the logarithm of the probability of two events occurring at the same time and the product of their respective probabilities of occurrence.

[0052]

[0053] Among them, P(x,y) represents the probability of events x and y occurring simultaneously, and P(x) and P(y) represent the marginal probabilities of x and y occurring independently.

[0054] If the PMI score is positive, it means that the probability of two events occurring at the same time is higher than expected; if the value is negative, it means it is lower than expected; in addition, if the value is zero, it means that the two events are independent.

[0055] In simple terms, PPMI is to truncate the part of PMI score that is less than zero and keep only non-negative values. i represents the i-th word, so the variables in the formula can be replaced, and PPMI can be expressed as:

[0056] PPMI(w i ,w j )=max{PMI(w i ,w j ),0}

[0057] Doing so can ignore negative values ​​and focus only on positive associations between events, helping to capture the semantic relationships between words. It is worth noting that PPMI, like PMI, is not symmetric and is not normalized for the frequency of events, which means that common words may have higher PPMI scores simply because they occur more frequently. To address these issues, various normalization techniques can be applied.

[0058] (2) Cosine similarity

[0059] Cosine Similarity is a measure of the degree of similarity between two vectors in direction. It determines their similarity by calculating the cosine value of the angle between the two vectors. The value range of cosine similarity is between -1 and 1. The closer the value is to 1, the more similar the directions of the two vectors are; the closer the value is to -1, the directions of the two vectors are completely opposite; the value is 0, which means that the two vectors are orthogonal, that is, there is no similarity.

[0060]

[0061] Among them, A and B represent w i and w j The word embedding representation is set as A ij =cos(A,B). Similar to PPMI, as the cosine similarity increases, the relationship between the two words becomes closer.

[0062] (3) Syntactic structure

[0063] The syntactic dependency tree (Syntactic DependencyTree), also known as the dependency syntactic analysis tree, is a tree structure used in natural language processing to represent the syntactic relationship between words in a sentence. In this structure, each word in a sentence is a node of the tree, and the edges between the nodes represent the dependency relationship between words. The syntactic dependency tree can reveal how the words in a sentence are interdependent and combined to form different syntactic structures of the sentence.

[0064] In the syntactic dependency tree, there is usually a special node, the root node, which represents the entire sentence. All other words are directly or indirectly dependent on the root node. Syntactic dependency relationships can be varied, such as subject-predicate relationship, verb-object relationship, attributive relationship, adverbial relationship, etc. These relationships are represented by directed edges, and the direction of the edge is usually from the dependent to the dependent.

[0065] Since the syntactic dependency tree can provide the syntactic relationship between words, this embodiment can use it as a metric value, and this embodiment uses this dependency structure as a matrix of the syntactic structure view.

[0066]

[0067] Among them, DT(w i ,w j ) is w i and w j When there is a dependency relationship between the relations in the syntactic structure, there will be edges connecting them. i ,w j ) has a larger value, the greater the weight of the edge between words.

[0068] Graph Learning Layer:

[0069] The original graph has an unstable topological structure, but it also contains a lot of useful information. Ideally, the learned graph structure can be used as a supplement to the original graph. Therefore, this embodiment assumes that the learned graph structure is a fine-tuning of the original graph, combines the learned graph structure with the original graph, and updates the graph.

[0070]

[0071] in, represents the adjacency matrix of the original graph after normalization at the i-th perspective, represents the degree matrix of the original graph, and They represent the two adjacency matrices of the original graph after the tth iteration. It is worth mentioning that is calculated based on the original node feature matrix X, where It is based on the previously updated node embedding This embodiment regards the final graph as a linear arrangement and integrates the advantages of both by balancing the parameter μ. Finally, GCN is used to map the node features of the original graph to the hidden layer.

[0072]

[0073] in, It is the vector representation of the t-th iteration under the i-th view of the implicit sentiment part, using the ReLU activation function.

[0074] Graph fusion layer:

[0075] This embodiment combines the advantages of three views and obtains a consistent document representation through a graphic fusion module.

[0076]

[0077] Among them, k∈[1,d] refers to the different dimensional representations of each document, and the function Concat represents a vector concatenation operation, the purpose of which is to fuse three different views into one vector. After obtaining logits, this embodiment obtains the vector representation of each document by average pooling. Next, this embodiment uses the Softmax layer of the σ function to decode the vector representation of each document.

[0078]

[0079] Finally, this embodiment obtains the graph fusion loss function L, which is used to reflect the difference between the true value y and the graph fusion predicted value y.

[0080]

[0081] Finally, the following experiment was carried out after the above steps.

[0082] Dataset: This example uses datasets such as Reddit and Pun-of-the-day. The dataset information is shown in Table 1.

[0083] Table 1 Dataset information

[0084]

[0085]

[0086] Evaluation indicators:

[0087] The evaluation indicators of this embodiment include accuracy (Acc), recall (Rec), and F1 score.

[0088] Baseline experiment:

[0089] The model proposed in this embodiment is mainly compared with SVM, Bi-LSTM-CNN, Bi-GRU, PACGA and other models. The experimental results of each model on the Puns dataset are shown in Table 2.

[0090] Table 2 Experimental results of the Puns dataset

[0091]

[0092] It can be seen from the table above that on the Puns dataset, the model of this embodiment achieved the best results in all three evaluation indicators. Compared with the previous optimal model PACGA, the KIG model is 0.52%, 0.96%, and 0.34% higher in precision, recall, and F1, respectively. Although the PACCG model can well represent the semantic information in the text, the knowledge fusion-based method proposed in this paper achieves better results in extracting key features with higher weights by comprehensively considering multiple knowledge sources.

[0093] On the Reddit dataset, this embodiment divides the dataset into Body (excluding the beginning and end), Punchline (the key part of the text), and Full (the entire content of the text), and calculates the accuracy rates respectively. The experimental results are shown in Table 3.

[0094] Table 3 Experimental results of the Reddit dataset

[0095]

[0096]

[0097] In Table 3, the results of the experiments using the Reddit dataset can be seen in this embodiment. This embodiment runs the model of this embodiment on the Body, Punchline, and Full parts respectively. On the Full dataset, this embodiment found that the accuracy of KIG was 67.84% and the accuracy of TEXTCNN was 67.10%. In order to learn more about the effect of the model on the dataset, this embodiment only uses the Body and Punchline datasets to see which part of the text is more useful for implicit emotion recognition. This embodiment found that most deep learning methods rely more on the Body part of the text in prediction, and this result may be due to the fact that the deep learning model can extract more features from the Body part. When considering global features, the accuracy of most deep learning models is higher than when there is only local data.

[0098] Ablation experiment:

[0099] In order to further evaluate the influence of each module of the model on its performance, this embodiment further studies the ablation experiment. First, this embodiment experiments on the single-view graph structure and the multi-view fusion graph structure to evaluate the contribution of the fusion graph structure to KIG. In addition, this embodiment also removes the iterative optimization module and the graph regularization module respectively to evaluate the influence of these two modules. The specific experimental results are shown in Table 4.

[0100] Table 4 Ablation experiment results

[0101]

[0102] In Table 4, w / o IL means no iterative learning, and w / o GR means graph regularization. This embodiment performs model ablation experiments on the Puns dataset and the Reddit dataset. By comparing the experimental results of the single-view graph structure and the multi-view fusion graph structure of the entire model, this embodiment can see that the multi-view fusion graph structure improves the performance of the model, which proves that the proposed knowledge fusion framework is helpful for the construction of the original graph. By turning off the iterative learning component, this embodiment can see that the performance of the model is significantly reduced, which proves the effectiveness of the proposed iterative learning framework for graph learning problems. This embodiment can also see the benefits of using graph regularization loss to jointly train the model.

[0103] Therefore, the framework in this embodiment can iteratively optimize the original graph structure to obtain an implicit graph structure that is more suitable for downstream implicit sentiment analysis tasks. It can also provide rich multi-source information and enhance the expressiveness of implicit emotions by fusing graph structures of different views.

[0104] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0105] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. An implicit sentiment analysis method based on iterative graph structure learning, characterized in that: include: Get the original graph structure of the target text; The original graph structure is input into the KIG model for optimization and fusion to obtain an implicit graph structure; The implicit graph structure is used to express implicit sentiment analysis; the KIG model includes a text encoder, a graph construction layer, a graph learning layer, and a graph fusion layer; The text encoder is used to use a dictionary to perform high-dimensional space mapping on implicit emotional text to obtain word vectors; the graph construction layer is used to construct multiple initial graph structures in different view forms based on the word vectors; The graph learning layer is used to extract features from each of the initial graph structures in an iterative manner; and the graph fusion layer is used to integrate features of each view.

2. The implicit sentiment analysis method based on iterative graph structure learning according to claim 1, characterized in that: The views include co-occurrence statistics, cosine similarity, and views of syntactic dependency tree construction.

3. The implicit sentiment analysis method based on iterative graph structure learning according to claim 2 is characterized in that: The operation expression of the co-occurrence statistics is: PPMI(in i ,In j )=max{PMI(in i ,In j ),0} Among them, PPMI represents positive point mutual information, which is a reweighted form of co-occurrence metric; w i represents the i-th word; w j represents the jth word; PMI represents point mutual information.

4. The implicit sentiment analysis method based on iterative graph structure learning according to claim 2, characterized in that: The operational expression of the cosine similarity is: Among them, CosineSimilarity represents cosine similarity; A and B represent w i and w j word embedding representation.

5. The implicit sentiment analysis method based on iterative graph structure learning according to claim 2, characterized in that: The operation expression of the syntactic dependency tree is: Among them, A ij The matrix representing the syntactic dependency structure; DT(w i ,w j ) indicates w i and w j Relationships in syntactic structures.

6. The implicit sentiment analysis method based on iterative graph structure learning according to claim 1, characterized in that: The operation expression of the graph learning layer is: in, is the vector representation of the t-th iteration under the i-th view of the implicit sentiment part, using the Relu activation function, GCN represents graph convolution, X represents the original node feature matrix, Represents the optimized graph after combining the learning graph with the original graph.

7. The implicit sentiment analysis method based on iterative graph structure learning according to claim 1, characterized in that: The operation expression of the graph fusion layer is: in, Refers to the value logit of pronoun j at viewpoint 1 after t rounds of iterations.

8. An implicit sentiment analysis system based on iterative graph structure learning, characterized in that: include: Data acquisition module, used to obtain the original graph structure of the target text; An optimization and fusion module is used to input the original graph structure into the KIG model for optimization and fusion to obtain an implicit graph structure; the implicit graph structure is used to express implicit sentiment analysis; the KIG model includes a text encoder, a graph construction layer, a graph learning layer, and a graph fusion layer; The text encoder is used to use a dictionary to perform high-dimensional space mapping on implicit emotional text to obtain word vectors; the graph construction layer is used to construct multiple initial graph structures in different view forms based on the word vectors; The graph learning layer is used to extract features from each of the initial graph structures in an iterative manner; and the graph fusion layer is used to integrate features of each view.

9. An electronic device, characterized in that: It includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the implicit sentiment analysis method based on iterative graph structure learning according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the implicit sentiment analysis method based on iterative graph structure learning as described in any one of claims 1 to 7.

Images