Dynamic graph structure learning framework based on semantic and structural information complementation

Abstract

The invention provides a dynamic graph structure learning framework based on complementation of semantics and structure information, the framework is composed of a graph construction module, a graph fusion module and a dynamic graph structure learning module, constructing an initial static graph structure of the text based on the syntactic dependency tree and a co-occurrence statistical method; the graph fusion module is used for enhancing information contained in graph structures by combining and constructing different graph structures; and the graph structure learning module is used for carrying out iterative learning to obtain a graph structure with higher robustness. According to the method, the static initial structure can be dynamically optimized, the influence of noise accumulation on the performance of the model is prevented, two graph structures are established for semantic and structural information of the text, information between the graph structures is complementary, and the information understanding ability of the implicit emotion text is improved.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and in particular relates to a dynamic graph structure learning framework based on the complementarity of semantic and structural information. Background Art

[0002] At present, the models used to process implicit sentiment analysis tasks mainly include graph convolutional network models (GCN) and graph neural network models (GNN). Although GCN performs well in processing image data, its internal mechanism is relatively complex, making the decision process of the model difficult to explain. This means that when GCN is used for implicit sentiment analysis tasks, it is difficult to explain the sentiment classification process of the model. In addition, GCN requires a fixed adjacency matrix and degree matrix when processing dynamic graphs. If the structure of the graph changes, GCN will not be able to effectively adapt to this change. For directed graphs, GCN cannot assign different weights to different neighbors, which limits its performance on directed graphs to a certain extent. Although GNN performs well in implicit sentiment analysis, there are still some problems. On the one hand, the graph structure constructed by implicit sentiment text is relatively simple, without comprehensive consideration of text information, and semantic understanding is difficult; on the other hand, the construction of the initial static graph structure relies on a lot of manual labor and professional knowledge, and it is impossible to correct the errors introduced in the knowledge. Summary of the invention

[0003] The purpose of the present invention is to provide a dynamic graph structure learning framework that is complementary to semantic and structural information.

[0004] The technical solution of the present invention is as follows:

[0005] A dynamic graph structure learning framework based on the complementarity of semantic and structural information, the framework consists of a graph construction module, a graph fusion module and a dynamic graph structure learning module, wherein the graph construction module, combining the semantic information and structural information of the text, constructs the initial static graph structure of the text based on the syntactic dependency tree and the co-occurrence statistics method respectively; the graph fusion module enhances the information contained in the graph structure by combining and constructing different graph structures; the graph structure learning module is used for iterative learning to obtain a more robust graph structure.

[0006] Furthermore, the method for constructing the syntactic dependency tree is as follows:

[0007] Specify the dependency relationship "the relationship where xi depends on xj is represented by Ei,j";

[0008]

[0009] DT(xi, xj) is the dependency relationship between xi and xj, and the weight between nodes depends on the value of DT(xi, xj).

[0010] Furthermore, the common statistical method is specifically:

[0011] When xi appears, the probability of the word pair (xi, xj) appearing is calculated using the maximum likelihood estimation method;

[0012]

[0013] Where #(xi) is the number of times xi appears, (xi,xj) is the number of times they appear together in a window of fixed size, let Aij = P MLE (xj|xi), the larger the value is, the closer the relationship between the two words is.

[0014] Furthermore, the method for enhancing the information contained in the graph structure by combining and constructing different graph structures is:

[0015] A merge =pooling(A)

[0016]

[0017] in, is the attention matrix of the edge with the same size as the adjacency matrix, ⊙ is the matrix dot product, pooling(A)=pooling(A1,A2,…,Ar), where Ai represents the adjacency matrix of different graph structures obtained from the graph construction module.

[0018] Furthermore, the graph structure learning module includes a static graph and a dynamic learning graph, specifically:

[0019] A (t) =βL (0) +(1-β){αf(A (t) )+(1-α)f(A (1) )

[0020] in, is the adjacency matrix of the initial static graph after normalization, A (0) and D (0) are the adjacency matrix and degree matrix of the initial state of the static graph, A (t) represents the adjacency matrix at the tth iteration, α is the balance parameter, and β is the hyperparameter.

[0021] Furthermore, the graph structure learning module also includes constructing a graph structure, dynamically learning by embedding the similarity of nodes, and the similarity measurement function is based on an attention mechanism;

[0022]

[0023] In the formula, represents a non-negative weight vector used to highlight the different dimensions of node embedding, ⊙ represents the multiplication operation between elements, Represents a node embedding.

[0024] Furthermore, the learning framework is optimized by a cross entropy loss function.

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] The present invention can not only dynamically optimize the static initial structure to prevent the accumulation of noise from affecting the performance of the model, but also establish two graph structures for the semantic and structural information of the text, so that the information between the graph structures is complementary, thereby improving the ability to understand the information of implicit emotional text. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings generally illustrate various embodiments by way of example and not limitation, and together with the description and claims, serve to illustrate the embodiments of the invention. Where appropriate, the same reference numerals are used throughout the drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the present apparatus or method.

[0028] Figure 1 A schematic diagram of the framework of the present invention is shown. DETAILED DESCRIPTION

[0029] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] The present invention provides a dynamic graph structure learning framework (SIF) that integrates text structure information and semantic information. Figure 1 As shown in the figure, SIF consists of three parts: graph construction module, graph fusion module and dynamic graph structure learning module. In the graph construction module, the initial static graph structure of the text is constructed based on the syntactic dependency tree and co-occurrence statistics method, combining the semantic information and structural information of the text. In the graph fusion module, the information contained in the graph structure is enhanced by combining and constructing different graph structures. Finally, iterative learning is performed in the dynamic graph structure learning module to obtain a more robust graph structure.

[0031] Graph building blocks:

[0032] For most natural language processing tasks, the typical input is a text sequence rather than a graph sequence. Using GNN to construct a graph structure from a text sequence is a difficult task. This paper uses two graph construction methods, namely, constructing a graph structure G = (V, E) based on semantic and structural information to construct a graph structure input for implicit sentiment analysis.i,j Represents word x i and x j The relationship is calculated through different measurement relationships, namely dependency parse trees and co-occurrence relationships.

[0033] (1) Structural information

[0034] The structural information of sentences in the text is used to enhance the original sequence data. The dependency graph is usually used to describe the dependency relationship between different entities. The extracted dependency relationship can be converted into a graph structure. The dependency graph construction method for extracting the parse tree given the input text is as follows.

[0035] Specify the dependency relationship "the relationship where xi depends on xj is represented by Ei,j".

[0036]

[0037] DT(x i ,x j ) is the dependency relationship between xi and xj, and the weight between nodes depends on DT(x i ,x j ) value.

[0038] (2) Semantic Information

[0039] Similarly, semantic information in text sequences can also be used to enhance the original sequence data. The goal of the co-occurrence graph is to capture the co-occurrence relationship between words in a sentence, and the goal of co-occurrence analysis is to find the similarity between words and discover the potential structure in the text. Based on co-occurrence analysis, the model can derive a low-dimensional dense vector to implicitly represent the syntactic and semantic features of the text.

[0040] The co-occurrence matrix describes how words appear at the same time and defines the co-occurrence relationship between them. The co-occurrence frequency is a classic method for calculating the word co-occurrence matrix. When xi appears, the maximum likelihood estimation method (MLE) is used to calculate the word pair (x i ,x j ) probability of occurrence.

[0041]

[0042] The above formula represents x i and x j The global relationship between i ) is x i The number of occurrences, (x i ,x j ) is the number of co-occurrences in a fixed-size window. Let A ij =P MLE (x j |xi ), the larger the value, the closer the relationship between the two words.

[0043] Graph fusion module:

[0044] Most static graph construction methods only consider specific types of relationships between nodes. Although the generated graph can also capture structural or semantic information in the text, their ability to generate other types of graph relationships is also limited. To solve this problem, the present invention combines multiple graphs together to construct a fusion graph to enrich the information in the graph.

[0045] The present invention hopes to combine the advantages of multiple graph structures into a fused graph, and merge the edges of these graphs into a single graph by pooling adjacency vectors, pooling(A)=pooling(A1,A2,...,Ar), where Ai represents the adjacency matrix of different graph structures obtained from the graph construction module.

[0046] A merge =pooling(A)

[0047]

[0048] in is the attention matrix of the edge with the same size as the adjacency matrix, and ⊙ is the matrix dot product. Finally, the present invention obtains a merged graph by pooling the adjacency vectors.

[0049] Graph structure learning module:

[0050] The construction of static graphs has certain prior knowledge, but it also has its flaws. First, it requires a lot of manual labor and professional knowledge to build a graph topology with good performance. Second, manually creating a graph structure is prone to errors. In addition, in the process of building the graph structure, errors will continue to accumulate, which will lead to a decline in model performance in the later stage. Finally, the knowledge of the graph construction process usually comes only from the field of natural language processing, which is not optimal for downstream prediction tasks.

[0051] In order to solve the above problems, the present invention uses professional domain knowledge to construct a graph structure, and then dynamically learns by embedding the similarity of nodes. The similarity measurement function is based on the attention mechanism.

[0052]

[0053] A represents a non-negative weight vector used to highlight the different dimensions of node embedding, and ⊙ represents the multiplication operation between elements. ij Dynamic learning is achieved by iteratively updating node embeddings.

[0054] Then the final graph structure is jointly optimized by the graph construction module and the graph representation learning module. In order to obtain better graph structure information, the present invention combines the static graph and the dynamic learning graph.

[0055] A (t) =βL (0) +(1-β){αf(A (t) )+(1-α)f(A (1) )

[0056] in is the adjacency matrix of the initial static graph after normalization, A (0) and D (0) are the adjacency matrix and degree matrix of the initial state of the static graph. (t) Represents the adjacency matrix at the tth iteration, and A is calculated based on the initial node feature m (1) , by iteratively updating the node embedding Z (t-1) Calculate A (t) The dynamic learning graph combines A (1) and A (t) The advantages of the word embedding matrix X are mapped to the latent space Z using the GCN.

[0057]

[0058] In the above formula, W is the weight matrix, ReLU is the activation function, and then we use the σ(·) function to represent the output function to obtain the predicted label.

[0059]

[0060] Finally, the model is optimized by minimizing the difference between the true value y and the predicted value through the cross entropy loss function.

[0061] L=-∑ynlogyn

[0062] This paper uses datasets such as Reddit, SemEval-2021 task 7 (SemEval), and Pun-of-the-day. The dataset information is shown in Table 1.

[0063] Table 1 Dataset information

[0064]

[0065] Evaluation indicators:

[0066] The evaluation indicators of the present invention include accuracy (Acc), recall (Rec), and F1 score.

[0067] The model proposed in this paper is mainly compared with TextCNN, RCNN, DPCNN, HAN, KIG and other models. The experimental results of each model on the Puns dataset are shown in Table 2.

[0068] Table 2 Experimental results of the Puns dataset

[0069]

[0070] As can be seen from Table 2, the performance of the SIF model of the present invention is better than other models. In addition, SIF is about 0.2% and 0.7% higher than the best HAN model in terms of accuracy and F1 overall evaluation indicators, respectively, which shows that SIF has higher accuracy on the Puns dataset. Although the HAN model uses a hierarchical attention network structure to effectively reflect the hierarchical structure of text, its ability to understand text semantics is poor, especially for some complex semantic relationships or context-related tasks, the HAN model needs stronger semantic understanding capabilities. The SIF model uses multiple graph structures to represent the semantic and structural information of sentences, which can better identify the sentiment characteristics of implicit text.

[0071] The experimental results on the SemEval dataset are shown in Table 3.

[0072] Table 3. Experimental results of SemEval dataset

[0073]

[0074] It can be seen from the above table that the SIF model proposed in the present invention is not only superior to the traditional text classification model, but also superior to the graph neural network models such as IDGL and KIG. Although KIG also uses multi-graph fusion technology, this process is performed after the graph learning module, which will cause some noise. The model proposed in the present invention performs dynamic learning after the graph fusion after the graph construction module, which can better reduce edge noise. The experimental results show that the model proposed in the present invention has a high recognition ability for implicit emotional text and can efficiently complete the emotion recognition task.

[0075] On the Reddit dataset, the present invention divides the dataset into Body (excluding the beginning and the end), Punchline (the key part of the text) and Full (the entire content of the text), and calculates the accuracy respectively. The experimental results are shown in Table 4.

[0076] Table 4 Experimental results of the Reddit dataset

[0077]

[0078] Ablation experiment:

[0079] In order to further evaluate the performance of the SIF model, the present invention conducted an ablation experiment. First, the present invention explored the influence of the structural information graph and the semantic information graph on the entire model. In addition, the experiment also considered the influence of the static initial graph and the dynamic learning graph on the algorithm. The experimental results are shown in Table 5.

[0080] Table 5 Ablation experiment results

[0081]

[0082] The present invention conducts ablation experiments to evaluate the impact of different model components. In Table 5, the semantic graph represents the result of the model using the semantic information graph alone, and the structural graph represents the result of the model using the structural information graph alone. The full model SIF refers to the result of using the semantic information graph and the structural information graph simultaneously. The w / o dynamic learning graph refers to the result of the model closing the dynamic learning process and using only the static fusion initial graph. As shown in Table 5, the experimental results of the single structural information graph structure, the single semantic information graph structure and the fusion graph structure are compared, and the results show that the full model SIF has the best effect, which indicates that the complementary mechanism of the structural information and the semantic information of the present invention helps the model to comprehensively extract the feature information of the text. The model SIF of the present invention solves the problem that most static graph construction methods in the past only consider specific relationships and are limited when using other types of graph relationships to a certain extent. SIF enriches the information in the graph by combining multiple graphs together to construct a fusion graph, providing a rich text graph structure with feature interaction for implicit sentiment analysis, which is more conducive to identifying implicit sentiment text feature information. In addition, the present invention finds that on the Puns and SemEval datasets, the performance of the model using only the semantic information graph is better than that using only the structural information graph, which indicates that the model pays more attention to the feature extraction of the semantic information graph on these two datasets. By turning off the dynamic iterative learning component and comparing the results with the entire model, the present invention can see that the performance of the model has dropped significantly, which proves the effectiveness of the proposed dynamic learning framework for graph learning problems. The model not only retains the information of the static initial graph, but also enhances the semantic understanding of the text by dynamically learning the structure of the graph. The present invention found that the results of turning off dynamic learning and using only the fused graph are worse than the results of using only a separate graph structure in some evaluation indicators, which may be because the fused graph without dynamic learning is superimposed with noise, which affects the performance.

[0083] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A dynamic graph structure learning framework based on the complementarity of semantic and structural information, characterized in that: The framework consists of a graph construction module, a graph fusion module and a dynamic graph structure learning module, wherein the graph construction module combines the semantic information and structural information of the text to construct the initial static graph structure of the text based on the syntactic dependency tree and the co-occurrence statistics method respectively; the graph fusion module enhances the information contained in the graph structure by combining and constructing different graph structures; the graph structure learning module is used to perform iterative learning to obtain a more robust graph structure.

2. The dynamic graph structure learning framework based on complementary semantic and structural information according to claim 1 is characterized in that: The method for constructing the syntactic dependency tree is as follows: Specify the dependency relationship "the relationship where xi depends on xj is represented by Ei,j"; DT(xi, xj) is the dependency relationship between xi and xj, and the weight between nodes depends on the value of DT(xi, xj).

3. The dynamic graph structure learning framework based on complementary semantic and structural information according to claim 1 is characterized in that: The common statistical method is specifically: When xi appears, the probability of the word pair (xi, xj) appearing is calculated using the maximum likelihood estimation method; Where #(xi) is the number of times xi appears, (xi,xj) is the number of times they appear together in a window of fixed size, let Aij = P MLE (xj|xi), the larger the value is, the closer the relationship between the two words is.

4. The dynamic graph structure learning framework based on complementary semantic and structural information according to claim 1 is characterized in that: The method for enhancing the information contained in the graph structure by combining and constructing different graph structures is: A merge =pooling(A) in, is the attention matrix of the edge with the same size as the adjacency matrix, ⊙ is the matrix dot product, pooling(A)=pooling(A1,A2,…,Ar), where Ai represents the adjacency matrix of different graph structures obtained from the graph construction module.

5. The dynamic graph structure learning framework based on complementary semantic and structural information according to claim 1 is characterized in that: The graph structure learning module includes a static graph and a dynamic learning graph, specifically: A (t) =βL (0) +(1-β){αf(A (t) )+(1-α)f(A (1) ) Among them, L (0) =D (0)-1 / 2 A (0) D (0)-1 / 2 is the adjacency matrix of the initial static graph after normalization, A (0) and D (0) are the adjacency matrix and degree matrix of the initial state of the static graph, A (t) represents the adjacency matrix at the tth iteration, α is the balance parameter, and β is the hyperparameter.

6. The dynamic graph structure learning framework based on complementary semantic and structural information according to claim 1 is characterized in that: The graph structure learning module also includes constructing a graph structure, dynamically learning by embedding the similarity of nodes, and the similarity measurement function is based on an attention mechanism; In the formula, represents a non-negative weight vector used to highlight the different dimensions of node embedding, ⊙ represents the multiplication operation between elements, Represents a node embedding.

7. The dynamic graph structure learning framework based on complementary semantic and structural information according to claim 1 is characterized in that: The learning framework is optimized by a cross entropy loss function.

Images