An aspect-level sentiment analysis method based on neighborhood window and knowledge graph enhancement

Abstract

The application relates to an aspect-level sentiment analysis method based on a neighborhood window and knowledge graph enhancement, and belongs to the technical field of sentiment analysis. The method aims to solve the problems of semantic disorder, insufficient utilization of external knowledge and ineffective fusion of semantic and syntactic information in existing methods. The method first encodes the input sentence and aspect word, and generates a preliminary semantic representation. Then, a syntactic dependency graph is constructed and a multi-copy random walk algorithm is used for enhancement, and a graph convolution network is used to extract syntactic information. Next, the neighborhood window mechanism and the opinion alignment module are used to capture aspect-related semantic information from different perspectives and generate multi-perspective sentiment representations. Finally, the syntactic information, semantic information and knowledge information are fused, and a classifier is used for sentiment polarity prediction. Experimental results show that the application can effectively improve the performance and precision of aspect-level sentiment analysis and enhance the adaptability of the model to different domain data.

Description

Technical Field

[0001] The present invention belongs to the technical field of sentiment analysis and relates to an aspect-level sentiment analysis method based on neighborhood windows and knowledge graph enhancement. Background Art

[0002] In recent years, with the rapid development of the internet, sentiment analysis has attracted considerable attention. Sentiment analysis tasks include document-level sentiment analysis, sentence-level sentiment analysis, and aspect-level sentiment analysis. Notably, aspect-level sentiment analysis has become a focus of current research due to its unique aspect-level nature. One of the core research issues in aspect-level sentiment analysis is determining the sentiment polarity of aspect words within a given sentence, including positive, negative, and neutral.

[0003] Early methods for aspect-level sentiment analysis primarily employed machine learning. However, in recent years, deep learning methods have been widely used in this field, with graph convolutional networks, in particular, becoming one of the most widely adopted techniques in deep learning. Although previous research has achieved significant success in aspect-level sentiment analysis, there is still room for improvement. For example, in the study of syntactic information, a random walk algorithm can be introduced to weight the syntactic dependency tree, highlighting aspect-related nodes. Secondly, existing research based on semantic information suffers from the widespread problem of semantic disorder. Building on existing techniques, a neighborhood window can be used to model the context around aspect words, and a multi-perspective aspect word attention matrix can be used to enhance the self-attention matrix. This allows the model to focus on aspect words and related opinion words while paying attention to the semantic structure of the sentence. In addition, existing research has insufficient utilization of external knowledge, which could be integrated into external knowledge graphs. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide an aspect-level sentiment analysis method based on neighborhood windows and knowledge graph enhancement. For aspect-level sentiment analysis tasks, there are semantic mismatches, insufficient utilization of external knowledge information, and a lack of integration of semantics, syntax, and external knowledge. A deep learning-based aspect-level sentiment analysis model is proposed. This model can incorporate external knowledge, alleviate the semantic disorder problem caused by the shortcomings of current attention methods, and effectively integrate semantic and syntactic information. Ultimately, the performance of the model is improved, and a model that can accurately identify the aspect-level sentiment polarity in sentences is obtained.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] An aspect-level sentiment analysis method based on neighborhood window and knowledge graph enhancement includes the following steps:

[0007] Step 1: Text information preprocessing, including:

[0008] Use the pre-trained language model to encode the input sentences and aspect words;

[0009] Use Bi-LSTM to generate preliminary semantic representation;

[0010] Step 2: Syntactic enhancement based on random walks, including:

[0011] Constructing a syntactic dependency graph;

[0012] Enhance the syntactic dependency graph using a multi-copy random walk algorithm;

[0013] The enhanced syntactic dependency graph is input into the graph convolution network for graph convolution operation to obtain the output of the syntactic module;

[0014] Step 3: Semantic enhancement based on neighborhood windows, including:

[0015] Get the neighborhood window, including:

[0016] Calculate the relative distance of each token to the aspect word;

[0017] Construct multiple neighborhood windows based on the window size threshold;

[0018] Use moving mask mechanism to generate mask vector;

[0019] Compute a representation of the neighborhood window;

[0020] The neighborhood window representation and the original context representation are concatenated and mapped back to a unified semantic space through a linear transformation to obtain the neighborhood window enhanced representation.

[0021] Using the viewpoint alignment module, the sentiment information in the neighborhood window enhanced representation is captured in parallel from different viewpoints to obtain multi-view sentiment representation;

[0022] Use a global average pooling layer to effectively combine multi-view sentiment representations into an integrated representation;

[0023] Step 4: Integrate external knowledge graphs, including:

[0024] Acquire knowledge embedding;

[0025] Concatenate the knowledge embedding with the output of Bi-LSTM;

[0026] Use the knowledge-aspect attention mechanism to obtain the representation of knowledge perspective;

[0027] Step 5: Feature fusion: concatenate the features obtained in steps 2, 3, and 4, and map them to a unified low-dimensional space through linear transformation to form a fused representation.

[0028] Step 6: Input the fusion representation into the classifier to predict the sentiment polarity.

[0029] Furthermore, the multi-copy random walk algorithm in step 2 includes:

[0030] Multiple copies: In the syntactic dependency graph, multiple copies are made for each aspect word node;

[0031] Random walk: The replica node of each aspect word moves along its syntactic dependency path until it reaches a leaf node in the syntactic dependency tree, and calculates the probability of each edge activation by measuring the likelihood that the syntactic dependency edge and the context word belong to the same subtree of the dependency tree.

[0032] Furthermore, the viewpoint alignment module in step three includes:

[0033] Utilize the abstract understanding representation converted from the sentence representation as the query vector;

[0034] Use a multi-head scaled dot-product attention mechanism to capture sentiment information in the neighborhood window enhanced representation from different perspectives in parallel;

[0035] Perform a linear transformation on each attention output vector to generate a three-dimensional emotion representation;

[0036] A global average pooling layer is used to effectively combine multi-view sentiment representations into an integrated representation.

[0037] Furthermore, the knowledge-aspect attention mechanism in step 4 includes:

[0038] Calculate the attention weights between context words and aspect words and knowledge embeddings respectively;

[0039] The attention weights of context words and aspect words are weighted and summed to obtain the knowledge-aspect attention weight;

[0040] The knowledge-aspect attention weights are used to perform weighted summation on the knowledge embeddings to obtain the representation of the knowledge perspective.

[0041] An aspect-level sentiment analysis system based on neighborhood windows and knowledge graph enhancement, including:

[0042] The text preprocessing module is used to encode the input sentences and aspect words and generate preliminary semantic representations;

[0043] The syntax enhancement module is used to construct a syntax dependency graph and enhance it using a multi-copy random walk algorithm. The enhanced syntax dependency graph is then input into the graph convolution network for graph convolution to obtain the output of the syntax module.

[0044] The semantic enhancement module is used for obtaining a neighborhood window, calculating a representation of the neighborhood window, splicing the neighborhood window representation and the original context representation, and mapping back to a unified semantic space through a linear transformation to obtain a neighborhood window enhanced representation, using a viewpoint alignment module to capture sentiment information in the neighborhood window enhanced representation from different perspectives in parallel to obtain a multi-perspective sentiment representation, and using a global average pooling layer to effectively combine the multi-perspective sentiment representation into an integrated representation.

[0045] The knowledge graph fusion module is used for obtaining a knowledge embedding, connecting the knowledge embedding with the output of the Bi-LSTM, and using a knowledge-aspect attention mechanism to obtain a knowledge perspective representation.

[0046] The feature fusion module is used for splicing the output of the syntax module, the output of the semantic enhancement module, and the output of the knowledge graph fusion module, and mapping to a unified low-dimensional space through a linear transformation to form a fusion representation.

[0047] The classification module is used for inputting the fusion representation into a classifier to predict the sentiment polarity.

[0048] The beneficial effects of the present application are that through multiple copy random walk, domain window enhancement, and external knowledge graph integration, the semantic disorder caused by the attention mechanism in the background technology, the insufficient use of external knowledge, and the inability to effectively fuse semantic and syntactic information are solved. Compared with the existing method, the present scheme not only improves the performance and precision, but also significantly reduces the computational cost through modular design, and enhances the adaptability to different domain data. The experimental results show that the performance of the model on the standard data set is better than that of the existing most advanced method, which provides an efficient and accurate solution for aspect-level sentiment analysis tasks, and fully realizes the invention target and expands the application scenarios of the model.

[0049] Other advantages, objects, and features of the present application will be set forth in part in the following specification, and in part will be apparent to those skilled in the art from the following description, or can be learned from practice of the present application. The objects and other advantages of the present application can be realized and obtained by the following description. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to make the objects, technical solutions and advantages of the present application clearer, the preferred detailed description of the present application will be combined with the drawings as follows, wherein:

[0051] Figure 1 The schematic diagram of the present application is shown in the figure. DETAILED DESCRIPTION

[0052] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0053] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0054] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0055] 1. System composition of the present invention

[0056] 1. Text preprocessing module

[0057] Input: Given a sentence-aspect pair (S,A), where S represents the input sentence and A represents the aspect word in the sentence.

[0058] Word embedding: Use the pre-trained language model BERT to encode the sentence S and aspect word A, and convert each word into a vector representation in a low-dimensional space.

[0059] Bi-LSTM: Two independent Bi-LSTM networks are used to process the word vector sequences of sentences and aspect words respectively to obtain the contextual representation of sentences and aspect words.

[0060] 2. Syntactic Enhancement Module Based on Random Walk

[0061] Construct a syntactic dependency graph: Based on the syntactic analysis results of sentence S, construct a syntactic dependency graph, where nodes represent words in the sentence and edges represent the dependency relationships between words.

[0062] Multi-copy random walk:

[0063] Multiple copies are made for each aspect word node, and the number of copies is equal to the number of downstream edges of the node.

[0064] Each replica node moves along its syntactic dependency path until it reaches a leaf node in the syntactic dependency tree.

[0065] The probability of each edge activation is calculated by measuring the likelihood that the syntactic dependency edge and the context word belong to the same subtree of the dependency tree.

[0066] Graph Convolutional Network: The enhanced syntactic dependency graph is input into the graph convolutional network for graph convolution operation to obtain the output of the syntactic module, which contains syntactic information.

[0067] 3. Semantic Enhancement Module Based on Neighborhood Window

[0068] Get the neighborhood window:

[0069] Calculate the relative distance of each token to the aspect word.

[0070] Construct multiple neighborhood windows based on a window size threshold.

[0071] A moving mask mechanism is used to generate a mask vector, which is used to select the word vector within each neighborhood window.

[0072] Calculate the representation of the neighborhood window: concatenate the word vectors in each neighborhood window to obtain the representation of the neighborhood window.

[0073] Neighborhood window enhanced representation: The neighborhood window representation and the original context representation are concatenated and mapped back to a unified semantic space through linear transformation to obtain the neighborhood window enhanced representation.

[0074] Viewpoint Alignment Module:

[0075] The abstract understanding representation converted from the sentence representation is used as the query vector.

[0076] A multi-head scaled dot-product attention mechanism is used to capture sentiment information in the neighborhood window enhanced representation from different perspectives in parallel.

[0077] Each attention output vector is linearly transformed to generate a three-dimensional emotion representation.

[0078] Integrating multi-view sentiment representations: Using a global average pooling layer, the multi-view sentiment representations are effectively combined into an integrated representation.

[0079] 4. External knowledge graph fusion module

[0080] Knowledge embedding: Obtain knowledge embedding using a knowledge graph embedding method based on semantic matching, denoted as sentence knowledge embedding and aspect word knowledge embedding, respectively.

[0081] Connection knowledge embedding: Connect the knowledge embedding with the output of Bi-LSTM to form sentence knowledge representation and aspect word knowledge representation.

[0082] Knowledge-aspect attention mechanism:

[0083] Calculate the attention weight between the context word and the aspect word and the knowledge embedding, respectively.

[0084] Weighted sum the attention weights of the context word and the aspect word to obtain the knowledge-aspect attention weight.

[0085] Weighted sum the knowledge embedding using the knowledge-aspect attention weight to obtain the representation of the knowledge perspective.

[0086] 5. Feature fusion module

[0087] Concatenate the output of the syntax module, the output of the semantic enhancement module, and the output of the knowledge graph fusion module, and map them to a unified low-dimensional space through linear transformation to form a fusion representation.

[0088] 6. Classification module

[0089] Input the fusion representation into a classifier (such as a fully connected layer) to predict the sentiment polarity and output the probability distribution of the sentiment polarity.

[0090] II. Working process and principles of the present application

[0091] As shown in Figure 1 , the process of the present application is as follows:

[0092] Step 1: Text information preprocessing

[0093] Given a sentence-aspect pair (S, A), use the commonly used pre-trained word embedding model BERT to represent each word in S and A, respectively. Convert each word x i into a vector in a low-dimensional space. This conversion is achieved through a mapping matrix , where |V| represents the size of the vocabulary, and dx represents the dimension of the word embedding. In this paper, the BERT encoder is used, and the input format is "[CLS] sentence [SEP] aspect [SEP]". This process generates the aspect-related context representation of the sentence H = h CLS ,h1,…,h n,These representations serve as input to subsequent modules, enabling the model to effectively extract and understand the semantic meaning of the input.

[0094] H=BERT(S)

[0095] After obtaining the embedding, two independent BiLSTMs are used to convert the word vector sequence into a context-level representation. In this way, the model can understand the contextual relationship of the entire sentence sequence and the aspect word sequence, and at the same time, the model can initially understand the relationship between the order of words in the sequence and the context. The hidden representation h of each word in the sentence and aspect word is si and h aj It can be obtained by the following formula:

[0096]

[0097] Among them, the forward propagation and backward propagation LSTM are respectively expressed as and The lengths of the sentence and aspect word are denoted as n and m, respectively. The semicolon ";" denotes a vector concatenation operation. When processing sequential data, LSTM captures information in the sequence by continuously updating the cell state and hidden state.

[0098] The embedded Bi-LSTM output of sentence S is represented as H S ={h s1 ,h s2 ,…,h si ,…,h sn}. Get the Bi-LSTM representation of aspect word A, denoted as H A =h a1 ,h a2 ,…,h ai ,…,h am .

[0099] Step 2.1: Random Walk-based Syntactic Enhancement

[0100] First, a dependency graph is constructed based on the dependency tree of each sentence. In the syntactic dependency graph, words with a dependency relationship are connected by edges, and each word in the sentence is considered a node in the syntactic dependency graph. More specifically, if two nodes are connected by an edge, the value is 1, otherwise it is 0. This gives the initial expression of the syntactic dependency graph, which is defined as G = V, E, A, where V is the set of word nodes, E is the set of dependency edges, and A is the basic syntactic adjacency matrix:

[0101]

[0102] Most previous studies have only constructed simple basic syntactic dependency graphs, but have overlooked scenarios where syntactic parsers may make parsing errors and have poor ability to perceive aspect-related syntactic information. Based on these two potential shortcomings of simple syntactic dependency graphs, a novel multi-copy random walk algorithm is proposed to enhance the syntactic dependency graph. The random walk algorithm still uses an undirected dependency graph, and the entire algorithm is divided into two steps: multi-copy copying and random walks. For aspect-level sentiment analysis tasks, only dependency edges or dependency paths rooted in aspect words are considered, while the remaining dependency edges still follow the basic syntactic dependency graph.

[0103] Multiple replicas: In a syntactic dependency graph, a sentence often contains multiple aspects, and an aspect can have multiple paths connecting to informative words in the dependency subtree. Therefore, a single replica cannot monitor all paths. To address this issue, an agent replicates a total of d replicas at each aspect node, where d is the number of downstream edges from the node. The replicas then proceed to each downstream edge and begin their journey.

[0104] Random Walk: Each replica node of an aspect word moves along its syntactic dependency path until it reaches a leaf node in the syntactic dependency tree. The probability of each edge activation is calculated by measuring the likelihood that the syntactic dependency edge and the context word belong to the same subtree of the dependency tree. The algorithm calculates the probability of moving from one node to another using the cosine similarity between node representations, as shown in the formula:

[0105]

[0106] Among them, v i and v j are the vector representations of nodes i and j respectively. The higher the similarity, the greater the probability of transfer between nodes.

[0107] At the same time, for different edge types in the syntactic dependency tree, the algorithm will assign different weights to the edges. For example, subject dependency (nsubj) may be more valuable to the aspect than modifier dependency (amod), so nsubj can be given a higher weight. Define the weight matrix W, different types of edges have different weights W e

[0108]

[0109] The denominator is a normalized denominator to ensure that the transition probabilities sum to 1.

[0110] The final graph is updated to G = V, E, A, where V is the word node and E is the dependency edge type. After the random walk algorithm, the edges on the path related to the aspect words are updated to the probability values ​​obtained by the random walk algorithm. The final adjacency matrix A is modified as follows:

[0111]

[0112] Next, the matrix A obtained after updating by the random walk algorithm is input into the GCN for graph convolution operation:

[0113]

[0114] Finally, the output of the syntax module is: H syn

[0115] Step 2.2: Neighborhood window-based semantic enhancement

[0116] Get the neighborhood window: First, in order to determine the position of each token relative to the aspect word inside the window, we calculate the relative distance d of each token relative to the aspect word. i , the calculation formula is:

[0117]

[0118] P i represents the position of the i-th token, i ranges from 0 to n, where n is the length of the input sentence. a+1 Indicates the starting position of the aspect, and m indicates the length of the aspect.

[0119] To model various combinations of contextual neighborhood words, multiple neighborhood windows are constructed based on a window size threshold L, ranging from 0 to L. This approach provides L+1 neighborhood windows. By constructing neighborhood windows of varying sizes, contextual information at varying distances from aspect words can be captured. A moving mask mechanism is employed, which generates a mask vector based on the relative distance between the token and the aspect word.

[0120] The mask vector V of each context word under the window size l∈[0,L] li Defined as:

[0121]

[0122] Where E∈R d is a vector of all 1s, 0∈R d Is a vector of all zeros. V li Represents the mask vector of the i-th tag under the window size of l. So far, the context area with the aspect as the center and the window length of L is obtained.

[0123] Then, we get the representation H of the lth neighborhood window l span∈R (n+1)×d , its window size is l, and the calculation formula is:

[0124] H lspan =M l ·H

[0125] Among them, the mask matrix M l ={V l0 ,V l1 ,…,V ln}∈R (n+1)×d By n+1 mask vectors V li It is composed of, which means the window size of the input sentence is l.

[0126] By generating neighborhood window representations of different spans, the contextual information of a sentence can be abstracted and summarized to varying degrees, providing the model with multi-level sentiment information. This multi-scale representation helps the model more accurately classify sentiment in different contexts.

[0127] The neighborhood window representation and the original context representation H are concatenated to obtain the neighborhood window enhanced representation H enhanced ∈R (n+1)×d , and then mapped back to a unified d-dimensional semantic space through a shared linear transformation, the calculation formula is:

[0128] H enhanced =W1(H lspan ⊙H)+b1

[0129] Where W1∈R d×2b are the weight and bias matrices in the linear transformation layer, and ⊙ represents the concatenation operation.

[0130] By integrating the neighborhood window representation with the original sentence representation, more representative sentiment information can be further extracted in a unified semantic space. This step ensures that the model can more comprehensively understand the context when processing complex sentence structures, thereby more accurately predicting sentiment polarity.

[0131] In order to accurately capture opinion words and alleviate the problem of semantic mismatch, we adopt a viewpoint alignment module. This module captures the neighborhood window enhanced representation H from different viewpoints in parallel. enhanced We use the emotional information from h s The abstract understanding obtained by conversion is represented by C l ∈R dAs query vector, rather than a given aspect as a class tag, this allows for an abstract understanding of all the tags in the sentence, providing a more comprehensive view. This ensures a more concentrated distribution of attention in the neighborhood window, as nearby words may contain more valuable information for sentiment classification:

[0132] C l =f(W2·h s +b2)

[0133] Q l =W Q ·C l +b Q

[0134] K l =W K ·H enhance +b K

[0135] V l =W V ·H rnhance +b V

[0136] Where W2∈R d×d and b2∈R d are the weight and bias matrices in the linear transformation layer. The activation function Tanh is represented by f(). W Q ∈R d ×d、W K ∈R d×d and W V ∈R d×d is the weight matrix, b Q ∈R d 、b K ∈R d and b V ∈R d is the bias matrix in the three linear transformation layers.

[0137] The multi-head scaling dot product attention calculation formula is:

[0138] y la =W h (head l1 ⊙head l2 ⊙…⊙head lh )

[0139] head li =Attention(Q li ,K li ,V li )

[0140]

[0141] where d k is a scaling factor to avoid the point product value between query and key vectors being too large, thus leading to instability of the softmax function. li , K li and V li are the query, key and value vectors of the i-th head, respectively. d and h are the dimension of the vectors and the total number of heads, respectively. In addition, denotes the weight matrix for linear transformation. The model learns different W matrices by learning QKV respectively, so as to learn more comprehensive multi-perspective and multi-dimensional semantic information in the sentence.

[0142] Each attention output vector y la ∈R d is linearly transformed on [y 0a ,…,y La ] to generate a three-dimensional sentiment representation y ls ∈R3, and the calculation formula is:

[0143] y ls =W3·y la +b3

[0144] where W3∈R 3×d and b3∈R3 are the weight and bias matrices in the linear transformation layer.

[0145] In order to integrate the sentiment representations obtained from different neighborhood span combinations, a global average pooling layer is used to combine the multi-perspective sentiment representations Y s =[y 0s ,…,y Ls ]∈R (L+1)×3

[0146] into an effective combination:

[0147] H sem =AvgPool1d(Y s )

[0148] where H sem ∈R3 is the integrated representation, and AvgPool1d() is a 1-dimensional average pooling layer with a kernel size of L+1.

[0149] Step 2.3: Incorporate external knowledge graph

[0150] First, obtain the knowledge embeddings, denoted as K S and K A . Then, connect the knowledge embeddings with the output of the Bi-LSTM. The knowledge embedding representations of the context and aspect words are denoted as (KH) S and (KH)A , as shown in the following equation:

[0151] (KH) S =(K S ;H S )

[0152] (KH) A =(K A ;H A )

[0153] The semicolon ";" indicates the connection

[0154] In order to effectively collect the context vocabulary related to aspect words, a new knowledge-aspect attention mechanism is adopted, as shown in the following equation:

[0155]

[0156] Finally, non-aspect words are masked and then average pooling is applied to obtain the knowledge perspective representation T.

[0157]

[0158] The three branches are trained in parallel to provide comprehensive information for the overall model to predict the results.

[0159] Step 3: Feature Fusion

[0160] The above features are concatenated along the feature dimension to form a preliminary fusion representation:

[0161] H Concat =[H syn ;H Sem ;T[

[0162] The concatenated features are compressed through linear transformation and mapped to a unified low-dimensional space:

[0163] H Compressed =ReLU(W fuse H Concat +b fuse )

[0164] Among them, W fuse and b fuse is a learnable parameter and ReLU is the activation function

[0165] Step 4: Output the classification results

[0166] Input the compressed features into the classifier (such as the fully connected layer) to predict the sentiment polarity:

[0167] y=Softmax(W class H Compressed+b class )

[0168] where y is the probability distribution of sentiment polarity, W class and b class are the classifier parameters.

[0169] Application scenario: sentiment analysis of e-commerce reviews

[0170] Example

[0171] Product reviews on e-commerce platforms are crucial for consumer purchasing decisions. By analyzing the sentiment polarity of reviews, e-commerce platforms can understand user evaluations of products, thereby improving products, optimizing recommendation systems, etc.

[0172] Suppose there is a dataset containing 10,000 product reviews, each review containing a sentence and corresponding aspect words.

[0173] For example:

[0174] Sentence: This phone has good camera performance and long battery life.

[0175] Aspect words: camera performance

[0176] Sentence: The delivery speed is very slow, and the packaging is very poor.

[0177] Aspect words: delivery

[0178] Implementation steps:

[0179] Data preprocessing: clean and label the review dataset, and input the sentence and aspect words into the model.

[0180] Model training: use the training set to train the model, adjust the model parameters, and make the model accurately predict the sentiment polarity of reviews.

[0181] Model evaluation: use the test set to evaluate the model, calculate the accuracy, recall rate, F1 value and other indicators of the model.

[0182] Model application: apply the trained model to actual scenarios, perform sentiment analysis on new reviews, and perform corresponding processing based on the analysis results.

[0183] Suppose we use the above method to analyze a new review:

[0184] Sentence: This computer has a clear screen and fast running speed, but the heat dissipation effect is not very good.

[0185] Aspect words: heat dissipation effect

[0186] Model output:

[0187] The screen is clear: positive

[0188] Running speed is also very fast: positive

[0189] The heat dissipation effect is not very good: negative

[0190] Result analysis:

[0191] Based on the model's output, we can see that users are generally satisfied with their computer's screen and speed, but less so with its cooling performance. E-commerce platforms can use this information to improve their products, such as optimizing cooling designs or recommending computers with better cooling performance.

[0192] Numerical example:

[0193] Training set: 8,000 reviews

[0194] Test set: 2000 reviews

[0195] Model parameters: BERT model, Bi-LSTM, GCN, multi-head attention mechanism, knowledge graph embedding method

[0196] Model evaluation indicators: accuracy, recall, F1 value

[0197] Expected results:

[0198] After training and evaluation, we expect the model to achieve high precision, recall, and F1 value, thereby enabling effective sentiment analysis of e-commerce reviews.

[0199] Experiments were conducted on standard datasets, and the results showed that the present invention can effectively improve the performance and accuracy of aspect-level sentiment analysis and enhance the model's adaptability to data from different fields.

[0200] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. An aspect-level sentiment analysis method based on neighborhood windows and knowledge graph enhancement, characterized by: The following steps are involved: Step 1: Text information preprocessing, including: Use the pre-trained language model to encode the input sentences and aspect words; Use Bi-LSTM to generate preliminary semantic representation; Step 2: Syntactic enhancement based on random walks, including: Constructing a syntactic dependency graph; Enhance the syntactic dependency graph using a multi-copy random walk algorithm; The enhanced syntactic dependency graph is input into the graph convolution network for graph convolution operation to obtain the output of the syntactic module; Step 3: Semantic enhancement based on neighborhood windows, including: Get the neighborhood window, including: Calculate the relative distance of each token to the aspect word; Construct multiple neighborhood windows based on the window size threshold; Use moving mask mechanism to generate mask vector; Compute a representation of the neighborhood window; The neighborhood window representation and the original context representation are concatenated and mapped back to a unified semantic space through a linear transformation to obtain the neighborhood window enhanced representation. Using the viewpoint alignment module, the sentiment information in the neighborhood window enhanced representation is captured in parallel from different viewpoints to obtain multi-view sentiment representation; Use a global average pooling layer to effectively combine multi-view sentiment representations into an integrated representation; The viewpoint alignment module in step 3 includes: Utilize the abstract understanding representation converted from the sentence representation as the query vector; Use a multi-head scaled dot-product attention mechanism to capture sentiment information in the neighborhood window enhanced representation from different perspectives in parallel; Perform a linear transformation on each attention output vector to generate a three-dimensional emotion representation; Use a global average pooling layer to effectively combine multi-view sentiment representations into an integrated representation; Step 4: Integrate external knowledge graphs, including: Acquire knowledge embedding; Concatenate the knowledge embedding with the output of Bi-LSTM; Use the knowledge-aspect attention mechanism to obtain the representation of knowledge perspective; Step 5: Feature fusion: concatenate the features obtained in steps 2, 3, and 4, and map them to a unified low-dimensional space through linear transformation to form a fused representation. Step 6: Input the fusion representation into the classifier to predict the sentiment polarity.

2. The aspect-level sentiment analysis method based on neighborhood window and knowledge graph enhancement according to claim 1 is characterized in that: The multi-copy random walk algorithm in step 2 includes: Multiple copies: In the syntactic dependency graph, multiple copies are made for each aspect word node; Random walk: The replica node of each aspect word moves along its syntactic dependency path until it reaches a leaf node in the syntactic dependency tree, and calculates the probability of each edge activation by measuring the likelihood that the syntactic dependency edge and the context word belong to the same subtree of the dependency tree.

3. The aspect-level sentiment analysis method based on neighborhood window and knowledge graph enhancement according to claim 1 is characterized in that: The knowledge-aspect attention mechanism in step 4 includes: Calculate the attention weights between context words and aspect words and knowledge embeddings respectively; The attention weights of context words and aspect words are weighted and summed to obtain the knowledge-aspect attention weight; The knowledge-aspect attention weights are used to perform weighted summation on the knowledge embeddings to obtain the representation of the knowledge perspective.

4. An aspect-level sentiment analysis system based on neighborhood windows and knowledge graph enhancement, applying the analysis method of claim 1, characterized in that: include: The text preprocessing module is used to encode the input sentences and aspect words and generate preliminary semantic representations; The syntax enhancement module is used to construct a syntax dependency graph and enhance it using a multi-copy random walk algorithm. The enhanced syntax dependency graph is then input into the graph convolution network for graph convolution to obtain the output of the syntax module. The semantic enhancement module is used to obtain a neighborhood window, calculate the neighborhood window representation, concatenate the neighborhood window representation and the original context representation, and map them back to a unified semantic space through a linear transformation to obtain the neighborhood window enhanced representation. The viewpoint alignment module is used to capture the sentiment information in the neighborhood window enhanced representation from different viewpoints in parallel to obtain a multi-view sentiment representation. The global average pooling layer is used to effectively combine the multi-view sentiment representations into an integrated representation. The knowledge graph fusion module is used to obtain knowledge embeddings, connect the knowledge embeddings with the output of Bi-LSTM, and use the knowledge-aspect attention mechanism to obtain the representation of knowledge perspectives; The feature fusion module is used to concatenate the outputs of the syntax module, the semantic enhancement module, and the knowledge graph fusion module, and map them to a unified low-dimensional space through linear transformation to form a fused representation; The classification module is used to input the fusion representation into the classifier to predict the sentiment polarity.

Images