Aspect-level sentiment analysis method based on neighborhood window and knowledge graph enhancement

Abstract

The invention relates to an aspect-level sentiment analysis method based on neighborhood windows and knowledge graph enhancement, and belongs to the technical field of sentiment analysis. The method aims at solving the problems that in an existing method, semantics are out of order, external knowledge is insufficient in utilization, and semantics and syntactic information cannot be effectively fused. According to the method, firstly, input sentences and aspect words are coded, and preliminary semantic representation is generated; then, a syntactic dependency graph is constructed, a multi-copy random walk algorithm is used for enhancement, and syntactic information is extracted in combination with a graph convolutional network; then, a neighborhood window mechanism and a viewpoint alignment module are used for capturing semantic information related to aspects from different view angles, and multi-view-angle emotion representation is generated; and finally, carrying out feature fusion on the syntactic information, the semantic information and the knowledge information, and inputting the information into a classifier to carry out sentiment polarity prediction. Experimental results show that the performance and precision of aspect-level sentiment analysis can be effectively improved, and the adaptability of the model to data in different fields is enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of sentiment analysis and relates to an aspect-level sentiment analysis method based on neighborhood window and knowledge graph enhancement. Background Art

[0002] In recent years, with the rapid development of the Internet, sentiment analysis tasks have attracted much attention. Sentiment analysis tasks include document-level sentiment analysis, sentence-level sentiment analysis, and aspect-level sentiment analysis. It is worth noting that aspect-level sentiment analysis has become the focus of current research due to its unique aspect-level characteristics. In the aspect-level sentiment analysis task, one of the core issues of the research is to determine the sentiment polarity of aspect words in a given sentence and the context of the aspect words in it, including positive, negative, and neutral.

[0003] Early aspect-level sentiment analysis methods mainly used machine learning methods. However, in recent years, deep learning methods have been widely used in this field, especially graph convolutional network methods have become one of the most widely adopted technologies in deep learning. Although previous studies have made remarkable achievements in aspect-level sentiment analysis tasks, there is still room for improvement. For example, in the study of syntactic information, a random walk algorithm can be introduced to weight the syntactic dependency tree and highlight the nodes related to the aspects. Secondly, the existing research based on semantic information generally has the problem of semantic disorder. Based on the existing technology, the neighborhood window can be used to model the context near the aspect words, and the multi-perspective aspect word attention matrix can be used to strengthen the self-attention matrix, so that the model can pay more attention to the aspect words and related opinion words while paying attention to the semantic structure of the sentence. In addition, the existing research has insufficient utilization of external knowledge, which can be integrated into the external knowledge graph. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide an aspect-level sentiment analysis method based on neighborhood windows and knowledge graph enhancement. For aspect-level sentiment analysis tasks, there are semantic mismatches, insufficient utilization of external knowledge information, and lack of integration of semantics, syntax, and external knowledge. A deep learning-based aspect-level sentiment analysis model is proposed. The model can incorporate external knowledge, alleviate the semantic disorder problem caused by the defects of current attention methods, and effectively integrate semantic and syntactic information. Ultimately, the performance of the model is improved, and a model that can accurately identify the aspect-level sentiment polarity in sentences is obtained.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] An aspect-level sentiment analysis method based on neighborhood window and knowledge graph enhancement includes the following steps:

[0007] Step 1: Text information preprocessing, including:

[0008] Encode the input sentences and aspect words using the pre-trained language model;

[0009] Use Bi-LSTM to generate preliminary semantic representation;

[0010] Step 2: Syntactic enhancement based on random walk, including:

[0011] Constructing syntactic dependency graphs;

[0012] Enhance the syntactic dependency graph using a multi-copy random walk algorithm;

[0013] The enhanced syntactic dependency graph is input into the graph convolution network for graph convolution operation to obtain the output of the syntactic module;

[0014] Step 3: Semantic enhancement based on neighborhood windows, including:

[0015] Get the neighborhood window, including:

[0016] Calculate the relative distance of each token to the aspect word;

[0017] Construct multiple neighborhood windows according to the window size threshold;

[0018] A moving mask mechanism is used to generate a mask vector;

[0019] Compute a representation of the neighborhood window;

[0020] The neighborhood window representation and the original context representation are concatenated and mapped back to a unified semantic space through a linear transformation to obtain the neighborhood window enhanced representation.

[0021] Using the viewpoint alignment module, the sentiment information in the neighborhood window enhanced representation is captured in parallel from different viewpoints to obtain multi-view sentiment representation;

[0022] Use a global average pooling layer to effectively combine multi-view sentiment representations into an integrated representation;

[0023] Step 4: Integrate external knowledge graphs, including:

[0024] Acquire knowledge embedding;

[0025] Connect the knowledge embedding with the output of Bi-LSTM;

[0026] Use the knowledge-aspect attention mechanism to obtain the representation of knowledge perspective;

[0027] Step 5: Feature fusion: concatenate the features obtained in steps 2, 3, and 4, and map them to a unified low-dimensional space through linear transformation to form a fused representation.

[0028] Step 6: Input the fusion representation into the classifier to predict the sentiment polarity.

[0029] Furthermore, the multi-copy random walk algorithm in step 2 includes:

[0030] Multiple copies: In the syntactic dependency graph, multiple copies are made for each aspect word node;

[0031] Random walk: The replica node of each aspect word moves along its syntactic dependency path until it reaches a leaf node in the syntactic dependency tree, and calculates the probability of each edge activation by measuring the likelihood that the syntactic dependency edge and the context word belong to the same subtree of the dependency tree.

[0032] Furthermore, the viewpoint alignment module in step three includes:

[0033] Using the abstract understanding representation converted from the sentence representation as the query vector;

[0034] Use multi-head scaled dot product attention mechanism to capture sentiment information in neighborhood window enhanced representation from different perspectives in parallel;

[0035] Linearly transform each attention output vector to generate a three-dimensional emotion representation;

[0036] A global average pooling layer is used to effectively combine multi-view sentiment representations into an integrated representation.

[0037] Furthermore, the knowledge-aspect attention mechanism in step 4 includes:

[0038] Calculate the attention weights between context words and aspect words and knowledge embeddings respectively;

[0039] The attention weights of context words and aspect words are weighted summed to obtain the knowledge-aspect attention weight;

[0040] The knowledge-aspect attention weights are used to perform weighted summation on the knowledge embeddings to obtain the representation of the knowledge perspective.

[0041] An aspect-level sentiment analysis system based on neighborhood window and knowledge graph enhancement, comprising:

[0042] The text preprocessing module is used to encode the input sentences and aspect words and generate preliminary semantic representations;

[0043] The syntax enhancement module is used to construct a syntax dependency graph and use a multi-copy random walk algorithm to enhance the syntax dependency graph. The enhanced syntax dependency graph is then input into the graph convolution network for graph convolution operation to obtain the output of the syntax module.

[0044] The semantic enhancement module is used to obtain the neighborhood window, calculate the representation of the neighborhood window, concatenate the neighborhood window representation and the original context representation, and map them back to a unified semantic space through a linear transformation to obtain the neighborhood window enhanced representation. The viewpoint alignment module is used to capture the sentiment information in the neighborhood window enhanced representation from different perspectives in parallel to obtain a multi-view sentiment representation. The global average pooling layer is used to effectively combine the multi-view sentiment representation into an integrated representation.

[0045] The knowledge graph fusion module is used to obtain knowledge embedding, connect the knowledge embedding with the output of Bi-LSTM, and use the knowledge-aspect attention mechanism to obtain the representation of knowledge perspective;

[0046] The feature fusion module is used to concatenate the output of the syntactic module, the output of the semantic enhancement module, and the output of the knowledge graph fusion module, and map them to a unified low-dimensional space through linear transformation to form a fusion representation;

[0047] The classification module is used to input the fusion representation into the classifier to predict the sentiment polarity.

[0048] The beneficial effects of the present invention are: through multi-copy random walks, domain window enhancement, external knowledge graph integration, etc., the semantic disorder caused by the attention mechanism in the background technology, the insufficient use of external knowledge, and the inability to effectively integrate semantic and syntactic information are solved. Compared with the existing methods, this solution not only improves the performance and accuracy, but also significantly reduces the computational cost through modular design, while enhancing the adaptability to data from different fields. Experimental results show that the model performs better than the existing most advanced methods on standard data sets, provides an efficient and accurate solution for aspect-level sentiment analysis tasks, fully realizes the invention goals and expands the application scenarios of the model.

[0049] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0051] Figure 1 This is a schematic diagram of the present invention. DETAILED DESCRIPTION

[0052] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0053] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0054] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0055] 1. Composition of the system of the present invention

[0056] 1. Text preprocessing module

[0057] Input: Given a sentence-aspect pair (S,A), where S represents the input sentence and A represents the aspect word in the sentence.

[0058] Word embedding: Use the pre-trained language model BERT to encode the sentence S and aspect word A, and convert each word into a vector representation in a low-dimensional space.

[0059] Bi-LSTM: Two independent Bi-LSTM networks are used to process the word vector sequences of sentences and aspect words respectively to obtain the contextual representation of sentences and aspect words.

[0060] 2. Random Walk-based Syntactic Enhancement Module

[0061] Construct a syntactic dependency graph: Based on the syntactic analysis results of sentence S, construct a syntactic dependency graph, where nodes represent words in the sentence and edges represent the dependency relationships between words.

[0062] Multi-copy random walk:

[0063] Multiple copies are made for each aspect word node, and the number of copies is equal to the number of downstream edges of the node.

[0064] Each replica node moves along its syntactic dependency path until it reaches a leaf node in the syntactic dependency tree.

[0065] The probability of each edge activation is calculated by measuring the likelihood that the syntactic dependency edge and the context word belong to the same subtree of the dependency tree.

[0066] Graph Convolutional Network: The enhanced syntactic dependency graph is input into the graph convolutional network for graph convolution operation to obtain the output of the syntactic module, which contains syntactic information.

[0067] 3. Semantic enhancement module based on neighborhood window

[0068] Get the neighborhood window:

[0069] Calculate the relative distance of each token with respect to the aspect word.

[0070] Construct multiple neighborhood windows based on a window size threshold.

[0071] A moving mask mechanism is used to generate a mask vector, which is used to select the word vector within each neighborhood window.

[0072] Calculate the representation of the neighborhood window: concatenate the word vectors in each neighborhood window to get the representation of the neighborhood window.

[0073] Neighborhood window enhanced representation: The neighborhood window representation and the original context representation are concatenated and mapped back to a unified semantic space through a linear transformation to obtain the neighborhood window enhanced representation.

[0074] Viewpoint Alignment Module:

[0075] The abstract understanding representation converted from the sentence representation is used as the query vector.

[0076] A multi-head scaled dot-product attention mechanism is used to capture sentiment information in the neighborhood window enhanced representation from different perspectives in parallel.

[0077] Each attention output vector is linearly transformed to generate a 3D sentiment representation.

[0078] Integrating multi-view sentiment representations: Using a global average pooling layer, the multi-view sentiment representations are effectively combined into an integrated representation.

[0079] 4. External knowledge graph fusion module

[0080] Knowledge embedding: The knowledge embedding is obtained using the knowledge graph embedding method based on semantic matching, which are recorded as sentence knowledge embedding and aspect word knowledge embedding respectively.

[0081] Concatenate knowledge embeddings: Concatenate the knowledge embeddings with the output of Bi-LSTM to form sentence knowledge representation and aspect word knowledge representation.

[0082] Knowledge-Aspect Attention Mechanism:

[0083] The attention weights between context words and aspect words and knowledge embeddings are calculated separately.

[0084] The attention weights of context words and aspect words are weighted summed to obtain the knowledge-aspect attention weight.

[0085] The knowledge-aspect attention weights are used to perform weighted summation on the knowledge embeddings to obtain the representation of the knowledge perspective.

[0086] 5. Feature Fusion Module

[0087] The output of the syntactic module, the output of the semantic enhancement module, and the output of the knowledge graph fusion module are concatenated and mapped to a unified low-dimensional space through linear transformation to form a fused representation.

[0088] 6. Classification module

[0089] The fused representation is input into a classifier (such as a fully connected layer), the sentiment polarity is predicted, and the probability distribution of the sentiment polarity is output.

[0090] 2. Working Process and Principle of the Invention

[0091] like Figure 1 As shown, the process of the present invention is as follows:

[0092] Step 1: Text information preprocessing

[0093] Given a sentence-aspect pair (S, A), use the commonly used pre-trained word embedding model BERT to represent each word in S and A respectively. i Convert to a vector in a low-dimensional space. This conversion is done by mapping the matrix Implementation, where |V| represents the vocabulary size and dx represents the dimension of word embedding. This paper uses the BERT encoder and follows the input format "[CLS] sentence [SEP] aspect [SEP]". This process generates a contextual representation of the sentence related to the aspect H = h CLS ,h1,…,h n, these representations serve as input to subsequent modules, enabling the model to effectively extract and understand the semantic meaning of the input.

[0094] H=BERT(S)

[0095] After obtaining the embedding, two independent BiLSTMs are used to convert the word vector sequence into a context-level representation. In this way, the model can understand the contextual relationship of the entire sentence sequence and the aspect word sequence, and the model can initially understand the relationship between the order of words in the sequence and the context. The hidden representation h of each word in the sentence and aspect word si and h aj It can be obtained by the following formula:

[0096]

[0097] Among them, the forward propagation and backward propagation LSTM are represented as and The lengths of sentences and aspect words are represented as n and m, respectively. The semicolon ";" represents a vector concatenation operation. When processing sequence data, LSTM captures information in the sequence by continuously updating the cell state and hidden state.

[0098] The embedded Bi-LSTM output of sentence S is represented as H S ={h s1 ,h s2 ,…,h si ,…,h sn}. The Bi-LSTM representation of aspect word A is obtained, denoted as H A =h a1 ,h a2 ,…,h ai ,…,h am .

[0099] Step 2.1: Random Walk-based Syntactic Enhancement

[0100] First, a dependency graph is constructed based on the dependency tree of each sentence. In the syntactic dependency graph, words with dependency relationships are connected by edges, and each word in the sentence is regarded as a node in the syntactic dependency graph. More specifically, once two nodes are connected by an edge, the value is 1, otherwise it is 0. Therefore, the expression of the initial syntactic dependency graph is obtained. This dependency graph is defined as G=V,E,A, where V is the set of word nodes, E is the set of dependency edges, and A is the basic syntactic adjacency matrix:

[0101]

[0102] Most previous studies have only constructed simple basic syntactic dependency graphs, but have ignored the scenarios where the syntactic parser may have parsing errors and the poor ability to perceive aspect-related syntactic information. Based on the two possible deficiencies of simple syntactic dependency graphs, a novel multi-copy mechanism random walk algorithm is proposed to enhance the syntactic dependency graph. For the random walk algorithm, an undirected dependency graph is still used. The entire algorithm step is divided into two parts: multi-copy copying and random walk. For aspect-level sentiment analysis tasks, only dependency edges or dependency paths with aspect words as roots are considered, while the remaining dependency edges still follow the basic syntactic dependency graph.

[0103] Multiple copies: In the syntactic dependency graph, since there are often multiple aspects in a sentence, and one aspect may have multiple paths connecting the information words on the dependency subtree, only one copy cannot focus on all paths. To solve this problem, an agent is used to copy a total of d copies at each aspect node, where d is the number of downstream edges of the node. The copies then turn to each downstream edge and start their journey.

[0104] Random walk: Each replica node of an aspect word moves along its syntactic dependency path until it reaches a leaf node in the syntactic dependency tree. The probability of each edge activation is calculated by measuring the likelihood that the syntactic dependency edge and the context word belong to the same subtree of the dependency tree. The algorithm calculates the probability of transferring from one node to another by the cosine similarity between node representations, as shown in the formula:

[0105]

[0106] Among them, v i and v j are the vector representations of nodes i and j respectively. The higher the similarity, the greater the probability of transfer between nodes.

[0107] At the same time, for different edge types in the syntactic dependency tree, the algorithm will assign different weights to the edges. For example, subject dependency (nsubj) may be more valuable to the aspect than modifier dependency (amod), so nsubj can be given a higher weight. Define the weight matrix W, different types of edges have different weights W e

[0108]

[0109] The denominator is a normalized denominator to ensure that the transition probabilities sum to 1.

[0110] The final graph is updated to G = V, E, A, where V is the word node and E is the dependency edge type. After the random walk algorithm, the edges on the path related to the aspect words are updated to the probability values ​​obtained by the random walk algorithm. The final adjacency matrix A is modified as follows:

[0111]

[0112] Next, the matrix A obtained after updating by the random walk algorithm is input into GCN for graph convolution operation:

[0113]

[0114] Finally, the output of the syntax module is: H syn

[0115] Step 2.2: Neighborhood window-based semantic enhancement

[0116] Get the neighborhood window: First, in order to determine the position of each token relative to the aspect word inside the window, we calculate the relative distance d of each token relative to the aspect word. i , the calculation formula is:

[0117]

[0118] P i represents the position of the i-th token, i ranges from 0 to n, where n is the length of the input sentence. a+1 indicates the starting position of the aspect, and m indicates the length of the aspect.

[0119] In order to model various combinations of contextual neighborhood words, multiple neighborhood windows are constructed according to the window size threshold L, where L ranges from 0 to L. This method provides L+1 neighborhood windows. By constructing neighborhood windows of different sizes, contextual information at different distances from aspect words can be captured. A moving mask mechanism is adopted, which generates a mask vector based on the relative distance between the token and the aspect words.

[0120] The mask vector V of each context word under the window size l∈[0,L] li Defined as:

[0121]

[0122] Among them, E∈R d is a vector of all 1s, 0∈R d is a vector of all zeros. li Represents the mask vector of the ith tag under the window size of l. So far, the context area with a window length of L centered on the aspect is obtained.

[0123] Then, we get the representation H of the lth neighborhood window l span∈R (n+1)×d , whose window size is l, and the calculation formula is:

[0124] H lspan =M l ·H

[0125] Among them, the mask matrix M l = {V l0 ,V l1 ,…,V ln}∈R (n+1)×d The n+1 mask vectors V li It is composed of, indicating that the window size of the input sentence is l.

[0126] By generating neighborhood window representations of different spans, the context information of the sentence can be abstracted and summarized to different degrees, thus providing the model with multi-level sentiment information. This multi-scale representation helps the model to classify sentiment more accurately in different contexts.

[0127] The neighborhood window representation and the original context representation H are concatenated to obtain the neighborhood window enhanced representation H enhanced ∈R (n+1)×d , and then mapped back to a unified d-dimensional semantic space through a shared linear transformation, the calculation formula is:

[0128] H enhanced =W1(H lspan ⊙H)+b1

[0129] Where W1∈R d×2b are the weight and bias matrices in the linear transformation layer, and ⊙ represents the concatenation operation.

[0130] By integrating the neighborhood window representation with the original sentence representation, more representative sentiment information can be further extracted in a unified semantic space. This step ensures that the model can more comprehensively understand the context when processing complex sentence structures, thereby more accurately predicting sentiment polarity.

[0131] In order to accurately capture opinion words and alleviate the problem of semantic mismatch, we adopt a viewpoint alignment module. This module captures the neighborhood window enhanced representation H from different viewpoints in parallel. enhanced We use the emotional information from h s The abstract understanding obtained by conversion is represented by C l ∈R dAs query vector, rather than given aspects as class tokens, this allows for an abstract understanding of all tokens in the sentence, providing a more comprehensive view. This ensures a more concentrated distribution of attention in the neighborhood window, as nearby words may contain more valuable information for sentiment classification:

[0132] C l =f(W2·h s +b2)

[0133] Q l =W Q ·C l +b Q

[0134] K l =W K ·H enhance +b K

[0135] V l =W V ·H rnhance +b V

[0136] Where W2∈R d×d and b2∈R d is the weight and bias matrix in the linear transformation layer. The activation function Tanh is represented by f(). W Q ∈R d ×d、W K ∈R d×d and W V ∈R d×d is the weight matrix, b Q ∈R d , b K ∈R d and b V ∈R d is the bias matrix in the three linear transformation layers.

[0137] The multi-head scaling dot product attention calculation formula is:

[0138] y la =W h (head l1 ⊙head l2 ⊙…⊙head lh )

[0139] head li =Attention(Q li ,K li ,V li )

[0140]

[0141] Among them, d k Q is a scaling factor used to prevent the dot product between the query and key vectors from being too large, which would lead to instability in the softmax function. li , K li and V li are the query, key, and value vectors of the ith head, respectively. d and h are the dimensions of the vector and the total number of heads, respectively. In addition, Represents the weight matrix used for linear transformation. The model obtains different W matrices by learning QKV separately, thereby learning the multi-view and multi-dimensional semantic information in the sentence more comprehensively.

[0142] Each attention output vector y la ∈R d In [y 0a ,…,y La ] to generate a three-dimensional emotion representation y ls ∈R3 calculation formula is:

[0143] y ls =W3·y la +b3

[0144] Among them, W3∈R 3×d and b3∈R3 are the weight and bias matrices in the linear transformation layer.

[0145] In order to integrate the sentiment representations obtained from different neighborhood span combinations, a global average pooling layer is used to transform the multi-view sentiment representation Y s =[y 0s ,…,y Ls ]∈R (L+1)×3

[0146] Effectively combines into:

[0147] H sem =AvgPool1d(Y s )

[0148] Among them, H sem ∈R3 is the integrated representation, AvgPool1d() is a 1-dimensional average pooling layer with a kernel size of L+1.

[0149] Step 2.3: Integrate into external knowledge graph

[0150] First, we get the knowledge embedding, denoted as K S and K A Subsequently, the knowledge embedding is concatenated with the output of the Bi-LSTM. The knowledge embedding representations of context and aspect words are referred to as (KH) S and (KH)A , as shown in the following equation:

[0151] (KH) S =(K S ;H S )

[0152] (KH) A =(K A ;H A )

[0153] The semicolon ";" indicates a connection

[0154] In order to effectively collect the context vocabulary related to aspect words, a new knowledge-aspect attention mechanism is adopted as shown in the following equation:

[0155]

[0156] Finally, non-aspect words are masked, and then average pooling is applied to obtain the knowledge perspective representation T.

[0157]

[0158] The three branches are trained in parallel to provide comprehensive information for the overall model to predict the results.

[0159] Step 3: Feature Fusion

[0160] The above features are concatenated along the feature dimension to form a preliminary fusion representation:

[0161] H Concat =[H syn ;H Sem ; T[

[0162] The concatenated features are compressed through linear transformation and mapped to a unified low-dimensional space:

[0163] H Compressed =ReLU(W fuse H Concat +b fuse )

[0164] Among them, W fuse and b fuse is a learnable parameter and ReLU is an activation function

[0165] Step 4: Output the classification results

[0166] The compressed features are input into the classifier (such as the fully connected layer) to predict the sentiment polarity:

[0167] y=Softmax(W class H Compressed+b class )

[0168] Among them, y is the probability distribution of sentiment polarity, W class and b class are the classifier parameters.

[0169] Application scenario: E-commerce review sentiment analysis

[0170] Example

[0171] Product reviews on e-commerce platforms are crucial to consumers’ purchasing decisions. By analyzing the sentiment polarity of reviews, e-commerce platforms can understand users’ evaluation of products and make product improvements, optimize recommendation systems, etc.

[0172] Suppose there is a dataset containing 10,000 product reviews, each of which contains a sentence and the corresponding aspect word.

[0173] For example:

[0174] Sentence: This phone takes great photos and has a long battery life.

[0175] Aspect word: Photo effect

[0176] Sentence: The delivery was very slow and the packaging was poor.

[0177] Aspect word: Express

[0178] Implementation steps:

[0179] Data preprocessing: Clean and annotate the review dataset, and input sentences and aspect words into the model separately.

[0180] Model training: Use the training set to train the model and adjust the model parameters so that the model can accurately predict the sentiment polarity of the comments.

[0181] Model evaluation: Use the test set to evaluate the model and calculate indicators such as the model's accuracy, recall, and F1 value.

[0182] Model application: Apply the trained model to actual scenarios, perform sentiment analysis on new comments, and take corresponding actions based on the analysis results.

[0183] Suppose we use the above method to analyze a new comment:

[0184] Sentence: This computer has a clear screen and runs very fast, but its heat dissipation effect is not very good.

[0185] Aspect word: heat dissipation effect

[0186] Model output:

[0187] The screen is clear: Positive

[0188] It also runs fast: positive

[0189] Cooling effect is not very good: negative

[0190] Result analysis:

[0191] Based on the model output, we can know that users are satisfied with the computer screen and running speed, but not satisfied with the heat dissipation effect. E-commerce platforms can make product improvements based on this result, such as optimizing the heat dissipation design or recommending computers with better heat dissipation performance to users.

[0192] Numerical example:

[0193] Training set: 8000 reviews

[0194] Test set: 2000 reviews

[0195] Model parameters: BERT model, Bi-LSTM, GCN, multi-head attention mechanism, knowledge graph embedding method

[0196] Model evaluation indicators: accuracy, recall, F1 value

[0197] Expected results:

[0198] After training and evaluation, we expect the model to achieve high precision, recall, and F1 value, thereby achieving effective sentiment analysis of e-commerce reviews.

[0199] Experiments were conducted on standard datasets, and the results showed that the present invention can effectively improve the performance and accuracy of aspect-level sentiment analysis and enhance the adaptability of the model to data from different fields.

[0200] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for aspect-level sentiment analysis based on neighborhood window and knowledge graph enhancement, characterized in that: The following steps are involved: Step 1: Text information preprocessing, including: Encode the input sentences and aspect words using the pre-trained language model; Use Bi-LSTM to generate preliminary semantic representation; Step 2: Syntactic enhancement based on random walk, including: Constructing syntactic dependency graphs; Enhance the syntactic dependency graph using a multi-copy random walk algorithm; The enhanced syntactic dependency graph is input into the graph convolution network for graph convolution operation to obtain the output of the syntactic module; Step 3: Semantic enhancement based on neighborhood windows, including: Get the neighborhood window, including: Calculate the relative distance of each token to the aspect word; Construct multiple neighborhood windows according to the window size threshold; A moving mask mechanism is used to generate a mask vector; Compute a representation of the neighborhood window; The neighborhood window representation and the original context representation are concatenated and mapped back to a unified semantic space through a linear transformation to obtain the neighborhood window enhanced representation. Using the viewpoint alignment module, the sentiment information in the neighborhood window enhanced representation is captured in parallel from different viewpoints to obtain multi-view sentiment representation; Use a global average pooling layer to effectively combine multi-view sentiment representations into an integrated representation; Step 4: Integrate external knowledge graphs, including: Acquire knowledge embedding; Connect the knowledge embedding with the output of Bi-LSTM; Use the knowledge-aspect attention mechanism to obtain the representation of knowledge perspective; Step 5: Feature fusion: concatenate the features obtained in steps 2, 3, and 4, and map them to a unified low-dimensional space through linear transformation to form a fused representation. Step 6: Input the fusion representation into the classifier to predict the sentiment polarity.

2. The aspect-level sentiment analysis method based on neighborhood window and knowledge graph enhancement according to claim 1 is characterized in that: The multi-copy random walk algorithm in step 2 includes: Multiple copies: In the syntactic dependency graph, multiple copies are made for each aspect word node; Random walk: The replica node of each aspect word moves along its syntactic dependency path until it reaches a leaf node in the syntactic dependency tree, and calculates the probability of each edge activation by measuring the likelihood that the syntactic dependency edge and the context word belong to the same subtree of the dependency tree.

3. The aspect-level sentiment analysis method based on neighborhood window and knowledge graph enhancement according to claim 1 is characterized in that: The viewpoint alignment module in step 3 includes: Using the abstract understanding representation converted from the sentence representation as the query vector; Use multi-head scaled dot product attention mechanism to capture sentiment information in neighborhood window enhanced representation from different perspectives in parallel; Linearly transform each attention output vector to generate a three-dimensional emotion representation; A global average pooling layer is used to effectively combine multi-view sentiment representations into an integrated representation.

4. The aspect-level sentiment analysis method based on neighborhood window and knowledge graph enhancement according to claim 1, characterized in that: The knowledge-aspect attention mechanism in step 4 includes: Calculate the attention weights between context words and aspect words and knowledge embeddings respectively; The attention weights of context words and aspect words are weighted summed to obtain the knowledge-aspect attention weight; The knowledge-aspect attention weights are used to perform weighted summation on the knowledge embeddings to obtain the representation of the knowledge perspective.

5. An aspect-level sentiment analysis system based on neighborhood window and knowledge graph enhancement, characterized by: include: The text preprocessing module is used to encode the input sentences and aspect words and generate preliminary semantic representations; The syntax enhancement module is used to construct a syntax dependency graph and use a multi-copy random walk algorithm to enhance the syntax dependency graph. The enhanced syntax dependency graph is then input into the graph convolution network for graph convolution operation to obtain the output of the syntax module. The semantic enhancement module is used to obtain the neighborhood window, calculate the representation of the neighborhood window, concatenate the neighborhood window representation and the original context representation, and map them back to a unified semantic space through a linear transformation to obtain the neighborhood window enhanced representation. The viewpoint alignment module is used to capture the sentiment information in the neighborhood window enhanced representation from different perspectives in parallel to obtain a multi-view sentiment representation. The global average pooling layer is used to effectively combine the multi-view sentiment representation into an integrated representation. The knowledge graph fusion module is used to obtain knowledge embedding, connect the knowledge embedding with the output of Bi-LSTM, and use the knowledge-aspect attention mechanism to obtain the representation of knowledge perspective; The feature fusion module is used to concatenate the output of the syntactic module, the output of the semantic enhancement module, and the output of the knowledge graph fusion module, and map them to a unified low-dimensional space through linear transformation to form a fusion representation; The classification module is used to input the fusion representation into the classifier to predict the sentiment polarity.

Images