An implicit sentiment analysis method based on text graph representation

Abstract

The application discloses a kind of implicit sentiment analysis methods based on text graph representation, belong to natural language processing field, including the following steps: S1: text pre-processing, then using pre-training word embedding model is encoded to target sentence and its context, obtain word embedding representation;S2: for the word in text, using external knowledge base matches common sense entity and obtains entity embedding representation by entity embedding method;S3: the semantic supplementary relationship of context and external knowledge to target sentence is modeled as text graph, the word embedding obtained in S1 and the entity embedding obtained in S2 are initialized as the node in graph;S4: pass through graph attention network in the text graph obtained in S3 Semantics information;S5: text graph representation is obtained using READOUT operation, and the sentiment probability distribution is calculated by combining softmax after passing through fully connected layer.The application aims to model the relationship between target sentence, context and external knowledge using text graph, supplement the text semantics at word level, improve the accuracy of implicit sentiment analysis, and provide technical support for the field of public opinion analysis.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of natural language processing, and in particular to an implicit sentiment analysis method based on text graph representation. BACKGROUND

[0002] Sentiment analysis is a popular direction of natural language processing, and has important value in understanding customer needs, improving products and services, and enhancing brand image. It provides enterprises and organizations with the ability to extract sentiment information from large amounts of text data, thereby helping to support data-driven decision-making and strategy formulation.

[0003] According to whether the text contains explicit sentiment words, the sentiment analysis task can be divided into explicit sentiment analysis and implicit sentiment analysis, where implicit sentiment analysis refers to mining sentiment tendencies from texts that lack obvious sentiment words. Research on text representation learning, natural language understanding, user modeling, and knowledge embedding all have a positive driving effect. With the standardization of network supervision, people tend to express their opinions in a subtle and implicit way, and the popularity of implicit sentiment analysis is gradually increasing. Compared with traditional explicit sentiment analysis, implicit sentiment analysis lacks sentiment words and is implicit in expression, and usually requires adding new features to the text semantics to enhance sentiment.

[0004] At present, existing implicit sentiment analysis methods mainly mine and fuse multiple features such as context and syntactic structure at the sentence level. This coarse-grained fusion method only supplements text semantics at the sentence level, ignoring the ambiguity of words in the text, and in some fine-grained scenarios, it is difficult to effectively represent text sentiment. The difference of the present application is to convert the sequence text into a graph structure to realize semantic interaction at the word level. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the purpose of the present application is to provide an implicit sentiment analysis method based on text graph representation; the method models the relationship between the target sentence, the context and the external knowledge using a text graph, supplements the text semantics at the word level, and improves the accuracy of implicit sentiment analysis.

[0006] In order to achieve the purpose of the present application, the following technical solutions are adopted:

[0007] An implicit sentiment analysis method based on text graph representation, comprising the following steps:

[0008] Step 1: Collecting text and pre-processing the text to obtain a main word set of the text, and then obtaining text word embedding representation through a pre-trained word embedding model.

[0009] Step 2: For the word set obtained in step 1, match the knowledge entities corresponding to the words in the external knowledge base, and then use the OpenKE tool set to obtain the entity embedding representation of the knowledge entities corresponding to the words.

[0010] Step 3: Convert the sequence text into a text graph with words as nodes, and link the knowledge entities corresponding to the words as extended nodes to the word nodes. The word embedding and entity embedding obtained in steps 1 and 2 are used as the initialization of the nodes in the text graph.

[0011] Step 4: For the text graph obtained in step 3, use the graph attention network to propagate semantic information between nodes to obtain a text graph with rich semantics.

[0012] Step 5: For the updated text graph in step 4, use the READOUT operation to obtain the text graph representation, map it to the sentiment space through the fully connected layer, and obtain the final sentiment orientation through softmax.

[0013] In step 1, the specific operations of preprocessing and word embedding are as follows:

[0014] Step 1.1: Text denoising. Denoise the original text, which includes deleting hyperlinks, converting traditional Chinese to simplified Chinese, converting full-width to half-width, deleting useless spaces, line breaks, and special symbols, etc.

[0015] Step 1.2: Tokenization. Use the jieba Chinese tokenization library to tokenize the denoised text and remove stop words to obtain the word set SW t and SW c .

[0016] Step 1.3: Word embedding. Use the MiniRBT pre-training model to perform word embedding on SW t and SW c to obtain the target sentence word embedding representation S t ={w1,w2,...,w tn}∈R tn×de and the context word embedding representation S c ={w1,w2,...,w cn}∈R cn ×de , where w represents the word vector, de is the word embedding dimension, tn is the number of words in the target sentence, and cn is the number of words in the context.

[0017] In step 2, the specific operations of knowledge matching and knowledge embedding are as follows:

[0018] Step 2.1: Knowledge matching. According to the word set SW t and SW cmatching the corresponding knowledge entity with conceptNet where KG i represents the word w i The set of entity relations matched with external knowledge, r is the relation between the word and the entity, t is the entity related to the word, and n is the number of entities.

[0019] Step 2.2: Knowledge embedding. The TransE method in the OpenKE tool set is used to obtain the embedding representation of the entity corresponding to the word in the sentence where T i represents the set of entity embeddings related to the word w i , e is the entity embedding representation, n is the number of entities, and dt is the dimension of the entity embedding.

[0020] The specific operation of the step 3 of text graph construction and text graph initialization is:

[0021] Step 3.1: Text graph construction. The relationship between the target sentence and its context and external knowledge is modeled as a text graph G={V,E}, where V represents the node set in the graph composed of word nodes and entity nodes, E is the edge set stored in a sparse matrix, and the edge set is composed of three types of edges: sliding window within the target sentence and context, full connection between context and target sentence, and connection between knowledge entity and its corresponding word.

[0022] Step 3.2: Text graph initialization. According to the word embedding S t and S c obtained in 1.3 and the entity embedding T i obtained in step 2.2, the nodes V in the text graph are initialized.

[0023] The specific operation of the step 4 of text graph updating is:

[0024] Step 4.1: Node contribution degree calculation. When the node semantic propagation is performed, an attention weight is assigned to each adjacent node of the current node, and then the contribution degree of each adjacent node to the current node is obtained through normalization operation. The calculation process is as follows:

[0025] e ij =α(LeakyReLU(W·[h i ||h j ])),j∈N i

[0026]

[0027] where h represents the node in the graph, N i represents the node h iThe set of adjacent nodes, LeakyReLU(·) is a non-linear activation function, and the α(·) operation maps a high-dimensional vector to a real number. ij It is the attention weight between two adjacent nodes, α ij h is obtained using the softmax(·) operation. i neighbor node h j The degree of contribution, W is a shared parameter.

[0028] Step 4.2: Node Update. Based on the contribution level and semantic information of neighboring nodes, the updated node is obtained through a learnable multi-head attention mechanism. The calculation process is as follows:

[0029]

[0030] Among them W k Here are the parameters for multi-head attention, where K is the number of attention heads, || is the vector concatenation operation, σ(·) is the non-linear activation function, and h is the multi-head attention parameter. j It is the node adjacent to the current node, α ij It is h j The extent of their contribution.

[0031] Step 4.3: Full graph update. Apply steps 4.1 and 4.2 to all nodes in the graph to obtain the updated text graph G' = {V', E}, where V' is the updated node and E is the edge information stored in a sparse matrix.

[0032] Specifically, the text graph representation and sentiment probability calculation in step 5 are performed as follows:

[0033] Step 5.1: Text Graph Representation. Based on the updated text graph G'={V',E} from Step 4, the text graph representation r∈R is obtained using the READOUT operation. d Where d represents the graph embedding dimension, the calculation process is as follows:

[0034] r=MeanPooling(V)+MaxPooling(V)

[0035]

[0036]

[0037] Where V represents the set of nodes in graph G, MeanPooling(·) is average pooling, MaxPooling(·) is maximum pooling, and h is the number of nodes in V.

[0038] Step 5.2: Sentiment Probability Calculation. The text graph representation is mapped to the sentiment space through a fully connected layer, and then the sentiment label is calculated using a softmax operation. The calculation process is as follows:

[0039]

[0040] wherein, wherein r is the resulting text graph representation vector, W and b are the learnable weights and bias respectively, and y represents the predicted sentiment label probability distribution.

[0041] In the training phase, according to the obtained sentiment label probability distribution and the real label, the model parameters are updated by using the Adma optimizer through the cross-entropy loss function to minimize the prediction error; in the prediction phase, the sentiment label with the maximum probability is taken as the predicted label.

[0042] Beneficial effects:

[0043] Compared with the prior art, the beneficial effects of the present application are embodied in:

[0044] (1) The present application proposes a text graph construction method, which models the association of target sentences with context and the connection of external knowledge with words as a graph. The originally abstract semantic relationship is converted into a specific graph structure, and the model can learn more accurate text representation.

[0045] (2) The present application uses a graph attention network to transmit semantic information between nodes and supplement text semantics at the word level. According to the semantic contribution degree of the nodes in the graph, different weights are assigned to different nodes in the semantic information transmission process by using the graph attention network, so as to focus on important information and obtain semantic complete and rich text graph representation. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the present application, the drawings used in the present application will be briefly introduced as follows.

[0047] Figure 1 Flow chart of implicit sentiment analysis method based on text graph representation;

[0048] Figure 2 Illustration of sliding window within a sentence;

[0049] Figure 3 Illustration of full connection between context and target sentence edges;

[0050] Figure 4 Illustration of connection between knowledge entity and word edges; DETAILED DESCRIPTION

[0051] In order to more clearly illustrate the technical solutions of the present application, the drawings used in the present application will be briefly introduced as follows.

[0052] Embodiment one: the present application provides a kind of implicit sentiment analysis method based on text graph representation, as shown in figure 1. Figure 1The specific steps are as follows:

[0053] S1: Collecting text and pre-processing the text to obtain the main word set of the text, and then obtaining the text word embedding representation through a pre-trained word embedding model.

[0054] S1.1: Read the text from the "smp2019-esisa" dataset, which mainly includes labels, target sentences and contexts. For the context and target sentence, perform data cleaning operations such as deleting hyperlinks, converting traditional Chinese to simplified Chinese, converting full-width to half-width, deleting useless spaces, line breaks and special symbols, etc.

[0055] S1.2: Use the jieba word segmentation library to perform word segmentation on the pre-processed text to obtain the word set SW t and SW c .

[0056] S1.3: Use the MiniRBT pre-training model to perform word embedding on SW t and SW c to obtain the target sentence word embedding representation S t ={w1,w2,...,w tn}∈R tn×de and the context word embedding representation S c ={w1,w2,...,w cn}∈R cn×de , where w represents the word vector, de is the word embedding dimension, tn is the number of words in the target sentence, and cn is the number of words in the context.

[0057] S2: Perform external knowledge matching on the words in the text, and then use the entity embedding tool to obtain the knowledge entity embedding representation.

[0058] S2.1: According to the words in the word set SW t and SW c obtained in S1.2, use conceptNet to match the corresponding knowledge entity where KG i represents the entity relationship set matched by the word w i with external knowledge, r is the relationship between the word and the entity, t is the entity related to the word, and n is the number of entities.

[0059] S2.2: Use the TransE method in the OpenKE tool set to obtain the embedding representation of the entities corresponding to the words in the sentence where T i represents the entity embedding set related to the word w i , e is the entity embedding representation, n is the number of entities, and dt is the entity embedding dimension.

[0060] S3: Convert the text into a graph structure, and the word embedding obtained in S1 and the entity embedding obtained in S2 are used as the initialization of the nodes in the graph.

[0061] S3.1: The relationship between the target sentence and its context and external knowledge is modeled as a text graph G = {V, E}, where V represents the node set in the graph composed of word nodes and entity nodes, and E is the edge set stored in a sparse matrix, which is composed of three types of edges, corresponding to: the sliding window inside the target sentence and its context (as shown in Figure 2 ), the full connection between the context and the target sentence (as shown in Figure 3 ), and the edge connection between the knowledge entity and its corresponding word (as shown in Figure 4 ).

[0062] S3.2: According to the word embedding obtained in S1 and the entity embedding obtained in S2, the word nodes and entity nodes in the text graph are initialized respectively.

[0063] S4: Use the graph attention network to propagate semantic information between nodes.

[0064] S4.1: Assign an attention weight to each adjacent node of the current node when propagating node semantics, and then obtain the contribution degree of each adjacent node to the current node through normalization operation. The calculation process is as follows:

[0065] e ij = a (LeakyReLU (W · [h i ||h j ]), j e N i

[0066]

[0067] where h represents a node in the graph, N i represents the set of adjacent nodes of the node h i , LeakyReLU(·) is a nonlinear activation function, and a(·) operation is to map a high-dimensional vector to a real number. e ij is the attention weight of the adjacent two nodes, a ij is the contribution degree of the neighbor node h i to h j , and W is a shared parameter.

[0068] S4.2: According to the contribution degree of the adjacent node and the semantic information of the adjacent node, an updated node is obtained through a learnable multi-head attention mechanism, and the calculation process is as follows:

[0069]

[0070] where W kis the multi-head attention parameter, K is the number of attention heads, || is the vector concatenation operation, σ(·) is the nonlinear activation function, h j is the node adjacent to the current node, α ij is the contribution degree of h j .

[0071] S4.3: Obtain the updated text graph G' = {V', E}, where V' is the updated node, and E is the edge information stored in the sparse matrix.

[0072] S5: Obtain the text graph representation through the READOUT operation, and then calculate the sentiment label probability distribution through the fully connected layer combined with the softmax.

[0073] S5.1: For the updated text graph G' = {V', E} in S4, the READOUT operation is used to obtain the text graph representation r ∈ R d , where d represents the graph embedding dimension, and the calculation process is as follows:

[0074] r = MeanPooling (V) + MaxPooling (V)

[0075]

[0076]

[0077] where V represents the node set of graph G, MeanPooling(·) is the average pooling, MaxPooling(·) is the maximum pooling, and h is a node in V.

[0078] S5.2: The text graph representation is mapped to the sentiment space through the fully connected layer, and then the sentiment label is calculated by the softmax operation, and the calculation process is as follows:

[0079]

[0080] where r is the obtained text graph representation vector, W and b are the learnable weights and bias respectively, and y represents the predicted sentiment label probability distribution.

[0081] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be regarded as limiting the claims involved.

[0082] Furthermore, it should be understood that although the description is made according to the embodiments, not every embodiment contains only one independent technical solution, and the description is made in this way only for the sake of clarity, and the person skilled in the art should consider the description as a whole, and the technical solutions in each embodiment can also be combined appropriately to form other embodiments that the person skilled in the art can understand.

Claims

1. An implicit sentiment analysis method based on text graph representation, characterized in that... Includes the following steps: Step 1: Collect text and preprocess it, including text noise reduction, word segmentation and word embedding; Step 2: For the word set obtained in Step 1, use an external knowledge base to match the corresponding knowledge entities, and then use the OpenKE tool to obtain the entity embedding representation of the knowledge entities; Step 3: Convert the sequence text into a text graph with words as nodes. The knowledge entities corresponding to the words are linked to the word nodes as extension nodes. The word embeddings and entity embeddings obtained in Step 1 and Step 2 are used as the initialization of nodes in the text graph. Step 4: For the text graph obtained in Step 3, use a graph attention network to propagate semantic information between nodes to obtain a semantically enriched text graph; Step 5: For the text graph updated in Step 4, use the READOUT operation to obtain the text graph representation, map it to the sentiment space through a fully connected layer, and obtain the final sentiment tendency through softmax. Step 4 includes: A graph attention network is used to propagate semantic information of nodes in the obtained text graph and update the nodes in the graph. The update process includes: Step 4.1: Calculation of Node Contribution; During aggregation, an attention weight is assigned to each neighboring node of the current node, and then a normalization operation is performed to obtain the contribution of each neighboring node to the current node. The calculation process is as follows: e ij =α(LeakyReLU(W [h i ||h j ])),j∈N i (1) Where h represents a node in the graph, N i Indicates the relationship with node h i The set of adjacent nodes, LeakyReLU(·) is a non-linear activation function, and the α(·) operation maps a high-dimensional vector to a real number. ij It is the attention weight between two adjacent nodes, α ij h is obtained using the softmax(·) operation. i neighbor node h j The degree of contribution, W is a shared parameter; Step 4.2: Node Update; Based on the contribution level and semantic information of neighboring nodes, the updated node is obtained through a learnable multi-head attention mechanism. The calculation process is as follows: Among them W k Here are the parameters for multi-head attention, where K is the number of attention heads, || is the vector concatenation operation, σ(·) is the non-linear activation function, and h is the multi-head attention parameter. j It is the node adjacent to the current node, α ij It is h j The degree of contribution; Step 4.3: Full graph update; Apply steps 4.1 and 4.2 to all nodes in the graph to obtain the updated text graph G' = {V', E}, where V' is the updated node and E is the edge information stored in a sparse matrix.

2. The implicit sentiment analysis method based on text graph representation according to claim 1, characterized in that, Step 1 includes: Step 1.1: Text Denoising; Denoising is performed on the unprocessed original text, specifically including removing hyperlinks, converting traditional Chinese to simplified Chinese, converting full-width characters to half-width characters, removing useless spaces and line breaks, and removing special characters; Step 1.2: Word segmentation; The jieba Chinese word segmentation library is used to segment the denoised text and remove stop words, resulting in word sets SW for the target sentence and its corresponding context. t and SW c ; Step 1.3: Word embedding; MiniRBT pre-trained model is used for SW t and SW c Word embedding is performed to obtain the target sentence word embedding representation S. t ={w1,w2,...,w tn }∈R tn×de and context word embeddings represent S c ={w1,w2,...,w cn }∈R cn×de , where w represents the word vector, de is the word embedding dimension, tn is the number of words in the target sentence, and cn is the number of words in the context.

3. The implicit sentiment analysis method based on text graph representation according to claim 2, characterized in that... Step 2 includes: Step 2.1: External knowledge matching; based on the word set SW obtained in Step 1.2 t and SW c The word w in i The external knowledge base ConceptNet is used to match the corresponding knowledge entities. KG i The word w i The set of entity relationships matched with external knowledge, where r is the relationship between words and entities, t is the entity related to the word, and n is the number of entities; Step 2.2: Entity Embedding; The TransE method from the OpenKE toolkit is used to obtain the embedded representations of the entities corresponding to the words in the sentence. Where T i Indicates the word w i The relevant set of entity embeddings, where e is the entity embedding representation, n is the number of entities, and dt is the entity embedding dimension.

4. The implicit sentiment analysis method based on text graph representation according to claim 3, characterized in that... Step 3 includes: Step 3.1: Text graph construction; The relationship between the target sentence, its context, and external knowledge is modeled as a text graph G = {V, E}, where V represents the set of nodes in the graph consisting of word nodes and entity nodes, and E is the set of edges stored in a sparse matrix. The specific set of edges consists of three types: sliding windows within the target sentence and context, fully connected edges between the context and the target sentence, and edge connections between knowledge entities and their corresponding words. Step 3.2: Text graph initialization; based on the word embeddings S obtained in Step 1.3 t and S c and the entity embedded in T obtained in step 2.2 i Initialize node V in the text graph.

5. The implicit sentiment analysis method based on text graph representation according to claim 1, characterized in that, Step 5 includes: Step 5.1: Text Graph Representation; Based on the updated text graph G'={V',E}, use the READOUT operation to obtain the text graph representation r∈R. d Where d represents the graph embedding dimension, the calculation process is as follows: r=MeanPooling(V)+MaxPooling(V) (4) Where V represents the set of nodes in graph G, MeanPooling(·) is average pooling, MaxPooling(·) is maximum pooling, and h is the number of nodes in V; Step 5.2: Sentiment Probability Calculation; The text graph representation is mapped to the sentiment space through a fully connected layer, and then the sentiment label is calculated using a softmax operation. The calculation process is as follows: Where r is the obtained text graph representation vector, and W and b are the learnable weights and biases, respectively. This represents the probability distribution of the predicted sentiment labels.

Images