Aspect-level sentiment analysis method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an aspect-level sentiment analysis method and device, electronic equipment and a storage medium, and relates to the technical field of natural language processing. Obtaining a to-be-analyzed statement, and inputting the to-be-analyzed statement into a pre-trained aspect-level sentiment classification model; performing word vector extraction on the to-be-analyzed statement by utilizing a plurality of serial coding layers in the text embedding layer to obtain a word vector sequence corresponding to the to-be-analyzed statement; the word vector sequence comprises word vectors output by each coding layer; inputting the word vector sequence into a semantic modeling layer for multi-level modeling and aspect attention analysis to obtain an emotion feature vector; and inputting the sentiment feature vector into a sentiment classification layer for sentiment analysis to obtain an aspect-level sentiment category corresponding to the to-be-analyzed statement. Therefore, the problem of semantic overfitting in the prior art can be solved, low-level semantics are fully utilized, and semantic information is fully utilized, so that the precision of aspect-level sentiment analysis is improved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to an aspect-level sentiment analysis method, device, electronic device, and storage medium. Background Art

[0002] Sentiment analysis is a key component of natural language processing and text analytics, focusing on identifying, extracting, and quantifying sentiment and opinion within text data. Sentiment analysis is widely used in various fields, including product feedback, personalized recommendation, market trend research, public opinion monitoring, financial services, social media content review, and customer service centers.

[0003] Traditional sentiment analysis typically focuses on the document and sentence levels, but with increasing research, more fine-grained aspect-based sentiment analysis (ABSA) is gaining increasing attention. The goal of aspect-based sentiment analysis is to identify specific aspects within a text and determine the sentiment polarity of each aspect. This fine-grained analysis can provide more detailed and accurate sentiment information for public opinion monitoring, decision support, product optimization, and other applications.

[0004] In the existing technology, there may often be a problem of semantic overfitting when performing aspect-level sentiment analysis, and the utilization of low-level semantics will be neglected, resulting in insufficient utilization of semantic information, which leads to low accuracy of aspect-level sentiment analysis. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide an aspect-level sentiment analysis method, device, electronic device and storage medium to solve the semantic overfitting problem existing in the prior art, and utilize low-level semantics to ensure the full utilization of semantic information, thereby improving the accuracy of aspect-level sentiment analysis.

[0006] In order to achieve the above objectives, the technical solutions adopted in the embodiments of the present application are as follows:

[0007] In a first aspect, the present application provides an aspect-level sentiment analysis method, the method comprising:

[0008] Obtaining a sentence to be analyzed, and inputting the sentence to be analyzed into a pre-trained aspect-level sentiment classification model; the aspect-level sentiment classification model includes a text embedding layer, a semantic modeling layer, and a sentiment classification layer;

[0009] Utilizing the serial multi-layer encoding layers in the text embedding layer to extract word vectors from the sentence to be analyzed, and obtaining a word vector sequence corresponding to the sentence to be analyzed; the word vector sequence includes the word vectors output by each layer of the encoding layer;

[0010] Inputting the word vector sequence into the semantic modeling layer for multi-level modeling and aspect attention analysis to obtain a sentiment feature vector;

[0011] The sentiment feature vector is input into the sentiment classification layer for sentiment analysis to obtain the aspect-level sentiment category corresponding to the sentence to be analyzed.

[0012] In an optional embodiment, the semantic modeling layer includes a vertical encoding layer, a horizontal multi-granularity layer, and an aspect attention layer;

[0013] The step of inputting the word vector sequence into the semantic modeling layer for multi-level modeling and aspect attention analysis to obtain a sentiment feature vector includes:

[0014] Inputting the word vector sequence into the vertical encoding layer, performing bidirectional modeling on the word vector sequence through the pyramid attention mechanism, and obtaining a pyramid graph corresponding to the word vector sequence;

[0015] Inputting the first feature representation corresponding to the root node in the pyramid graph and the target word vector in the word vector sequence into the horizontal multi-granularity layer for multi-scale modeling to obtain a second feature representation; the target word vector is the word vector output by the last encoding layer;

[0016] The second feature representation and the target word vector are input into the aspect attention layer for aspect attention analysis to obtain a sentiment feature vector.

[0017] In an optional embodiment, inputting the word vector sequence into the vertical encoding layer, performing bidirectional modeling on the word vector sequence through a pyramid attention mechanism, and obtaining a pyramid graph corresponding to the word vector sequence includes:

[0018] Inputting the word vector sequence into the vertical coding layer, convolving the word vector sequence layer by layer according to a preset convolution kernel size and a preset step size, and performing gate processing on the convolution operation results of each layer to obtain an initial pyramid map;

[0019] Determine the focus node corresponding to each of the nodes in the initial pyramid graph according to a first preset formula;

[0020] According to the focus node corresponding to each of the nodes and the second preset formula, the attention value corresponding to each of the nodes is calculated, and the feature representation of each node in the initial pyramid diagram is updated according to the attention value of each of the nodes to obtain the pyramid diagram corresponding to the word vector sequence.

[0021] In an optional implementation manner, the first preset formula is:

[0022]

[0023] in, Represents the attention node corresponding to the m-th node in the j-th layer, Represents the neighboring nodes of the mth node in the jth layer, Represents the child nodes of the mth node in the jth layer, Represents the parent node of the mth node in the jth layer, represents the mth node in the jth layer, A represents the number of neighboring nodes in the same layer, T represents the length of the first-layer node sequence in the pyramid graph, e represents the value range of m, and C represents the number of child nodes.

[0024] In an optional implementation manner, the second preset formula is:

[0025]

[0026] Among them, q i The query vector representing the i-th node in the pyramid graph, k m The key vector representing the mth node in the jth layer, d k Characterize the characteristic dimension of the node, v m The value vector representing the m-th node in the j-th layer.

[0027] In an optional embodiment, inputting the first feature representation corresponding to the root node in the pyramid graph and the target word vector in the word vector sequence into the horizontal multi-granularity layer for multi-scale modeling to obtain the second feature representation includes:

[0028] Inputting the first feature representation and the target word vector into the horizontal multi-granularity layer, processing the first feature representation through multiple parallel one-dimensional convolutional layers, and performing gate processing on the output results of each of the one-dimensional convolutional layers to obtain multiple convolution branch results;

[0029] The multiple convolution branch results are selectively learned through the Sigmoid function to obtain a selective learning result, and the selective learning result and the target word vector are processed through the residual connection and the ReLU activation function to obtain the second feature representation.

[0030] In an optional embodiment, inputting the second feature representation and the target word vector into the aspect attention layer for aspect attention analysis to obtain the sentiment feature vector includes:

[0031] Inputting the second feature representation and the target word vector into the aspect attention layer, processing the target word vector through a mask mechanism and a pooling operation to obtain a third feature representation;

[0032] The second feature representation and the third feature representation are interacted with each other through a third preset formula to obtain the emotion feature vector.

[0033] In an optional implementation, the third preset formula is:

[0034]

[0035] Among them, s i Characterizes the i-th feature component in the second feature representation, Represents the third feature representation, r represents the emotional feature vector, β i Characterizations i and The cosine similarity of α i Characterization of β i Attention weights normalized by the softmax function.

[0036] In a second aspect, the present application provides an aspect-level sentiment analysis device, the method comprising:

[0037] An acquisition module is used to acquire a sentence to be analyzed and input the sentence to be analyzed into a pre-trained aspect-level sentiment classification model; the aspect-level sentiment classification model includes a text embedding layer, a semantic modeling layer, and a sentiment classification layer;

[0038] A model processing module is used to extract word vectors from the sentence to be analyzed using the serial multi-layer encoding layers in the text embedding layer to obtain a word vector sequence corresponding to the sentence to be analyzed; the word vector sequence includes the word vectors output by each layer of the encoding layer;

[0039] The model processing module is further configured to input the word vector sequence into the semantic modeling layer for multi-level modeling and aspect attention analysis to obtain a sentiment feature vector;

[0040] The model processing module is further configured to input the emotion feature vector into the emotion classification layer for emotion analysis, and obtain the aspect-level emotion category corresponding to the sentence to be analyzed.

[0041] In a third aspect, the present application provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program that can be executed by the processor, and the processor can execute the computer program to implement the method described in any one of the aforementioned embodiments.

[0042] In a fourth aspect, the present application provides a storage medium having a computer program stored thereon, which implements the method described in any one of the aforementioned embodiments when the computer program is executed by a processor.

[0043] The aspect-level sentiment analysis method, device, electronic device and storage medium provided in the embodiment of the present application input the sentence to be analyzed into a pre-trained aspect and sentiment classification model for processing. The aspect-level sentiment classification model includes a text embedding layer, a semantic modeling layer and a sentiment classification layer. Among them, the serial multi-layer encoding layer in the text embedding layer can be used to extract word vectors for the sentence to be analyzed, thereby obtaining a sequence of word vectors corresponding to the sentence to be analyzed, that is, the word vectors output by each encoding layer. Based on this, the semantic modeling layer can be used to perform multi-level modeling and aspect attention analysis on the word vectors output by each encoding layer to obtain a sentiment feature vector, and the sentiment classification layer is used to process the sentiment feature vector to determine the aspect-level sentiment category corresponding to the sentence to be analyzed. In this way, the word vectors output by each encoding layer in the text embedding layer can be fully utilized during the model processing process, thereby solving the problem of semantic overfitting. At the same time, the method can also make full use of low-level semantics and effectively give play to the different advantages of each encoding layer in encoding semantic information, thereby ensuring the full utilization of semantic information and thus improving the accuracy of aspect-level sentiment analysis.

[0044] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 A block diagram of an electronic device provided in an embodiment of the present application is shown;

[0047] Figure 2 A schematic diagram of a process flow of the aspect-level sentiment analysis method provided in an embodiment of the present application is shown;

[0048] Figure 3 Shows a schematic diagram of the framework of the aspect-level sentiment analysis model;

[0049] Figure 4 Shown is a schematic diagram of the pyramid attention mechanism;

[0050] Figure 5 shows a schematic diagram of node connections;

[0051] Figure 6 Schematic diagram of convolution operation is shown;

[0052] Figure 7A schematic diagram showing the impact of the number of pyramid attention layers on model performance;

[0053] Figure 8 A schematic diagram showing the impact of the number of pyramid attention heads on model performance;

[0054] Figure 9 A schematic diagram showing the impact of different convolution scales on model performance;

[0055] Figure 10 A functional module diagram of an aspect-level sentiment analysis device provided in an embodiment of the present application is shown.

[0056] Icon: 100 - memory; 110 - processor; 120 - communication module; 200 - acquisition module; 210 - model processing module. DETAILED DESCRIPTION

[0057] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0058] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present application.

[0059] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0060] Sentiment analysis is a key branch of natural language processing (NLP) and text analytics, focusing on identifying, extracting, and quantifying sentiment and opinion within text data. Sentiment analysis is widely used in various fields, including product feedback, personalized preference recommendations, market trend research, public opinion monitoring, and financial services. For example, social media content moderation requires reviewing the sentiment of user-posted content to prevent malicious content from being successfully published. Customer service scenarios, such as telecommunications customer service centers and ETC customer service centers, also require sentiment analysis of customer-submitted text to optimize service processes.

[0061] Currently, traditional sentiment analysis typically focuses on the document and sentence levels. However, as research deepens, researchers have discovered that coarse-grained sentiment analysis is unable to handle texts with multiple facets and emotions. Consequently, finer-grained aspect-level sentiment analysis is gaining increasing attention. The goal of aspect-level sentiment analysis is to identify specific aspects within a text and determine the sentiment polarity of each aspect. This fine-grained analysis can provide more detailed and accurate sentiment information for public opinion monitoring, decision support, product optimization, and other applications.

[0062] For example, for the sentence "The servers have powerful computing power, but the rental price is expensive.", the contextual sentiment word corresponding to the aspect word "computing power" is the positive "powerful", while the contextual sentiment word corresponding to the aspect word "price" is the negative "expensive". Different aspects of the same sentence contain completely opposite emotional attitudes.

[0063] Early aspects of sentiment analysis primarily combined sentiment lexicons, grammatical rules, and machine learning. Through feature engineering, they transformed sentiment analysis into a classification task that could be handled by machine learning. However, these approaches relied heavily on feature engineering and ignored the inherent connections between features, limiting the model's generalization and adaptability. Currently, deep learning methods have achieved significant advantages in the field of aspect-level sentiment analysis. Their main idea is to use deep neural networks to build a model that can automatically learn sentiment-related features from large-scale text data and accurately predict sentiment classification for specific aspects of the text.

[0064] For example, the Long Short-Term Memory (LSTM) network effectively solves the gradient vanishing and gradient exploding problems of traditional Recurrent Neural Network (RNN) when dealing with long-term dependencies by introducing memory units, constant error propagation structures, and gating units. The Time-Dependent Long Short-Term Memory (TD-LSTM) network is used to handle target-dependent sentiment classification problems and predicts sentiment polarity by modeling the context before and after the target word respectively. Word embedding (word2vec), part-of-speech tags (POS), and contextual position information are crucial for accurately extracting feature-oriented sentiment classification. Therefore, this information can be integrated by constructing a multi-channel gated recurrent neural network (MC-GRU) and an attention mechanism to achieve high-precision aspect extraction and sentiment classification. In addition, there are many other model processing methods.

[0065] However, most of the above methods use static word embedding models that are insensitive to context, such as GloVe and Word2Vec. These models cannot effectively solve the problem of polysemy and are therefore not applicable in rapidly changing language environments.

[0066] With the advancement of deep learning technology, pre-trained language models such as Bidirectional Encoder Representations from Transformers (BERT), Generative Pre-trained Transformer (GPT), and Decoding-enhanced BERT with disentangled attention (DeBERTa) have gradually entered the field of natural language processing, greatly improving the accuracy and generalization ability of the ABSA model.

[0067] The key idea behind pre-trained language models is to pre-train them in an unsupervised manner on large-scale corpus data, and then use this pre-trained model to fine-tune various downstream natural language processing tasks. The word vector representations in pre-trained language models are dynamically adjusted based on the context, enabling them to handle polysemy and contextual dependencies.

[0068] In recent years, the method based on the BERT pre-trained language model has become one of the mainstream research methods for aspect-level sentiment analysis. The idea is to use sentiment analysis as a downstream task of BERT and fine-tune it by constructing different downstream structures to achieve a significant improvement in model performance.

[0069] However, in related technologies, when processing text through BERT, only the output of the last encoding layer is used. In BERT, the lower encoding layers primarily capture structured information such as phrases and syntax, while the higher encoding layers focus on encoding abstract semantic information. Therefore, if only the output of the last encoding layer is used, the semantic information of the lower encoding layers will be ignored, resulting in insufficient utilization of semantic information. In some cases, the information in the lower encoding layers already provides a good semantic representation. In this case, using only the output of the last encoding layer will lead to over-parameterization and fitting risks, resulting in inaccurate sentiment analysis. For example, in text classification tasks, the model is unable to capture keyword differences due to over-abstraction and instead relies on irrelevant features in the data for classification.

[0070] Based on this, the embodiments of the present application provide an aspect-level sentiment analysis method, device, electronic device and storage medium to solve the above technical problems.

[0071] Specifically, Figure 1 This is a block diagram of an electronic device provided in an embodiment of the present application, see Figure 1 The electronic device includes a memory 100, a processor 110, and a communication module 120. The memory 100, the processor 110, and the communication module 120 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines.

[0072] The memory 100 is used to store computer programs or data that can be executed by the processor. The memory 100 can be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0073] The processor 110 is used to read / write data or computer programs stored in the memory and execute the computer program to implement the aspect-level sentiment analysis method provided in the embodiment of the present application.

[0074] The communication module 120 is used to establish a communication connection between the electronic device and other communication terminals through a network, and to send and receive data through the network.

[0075] It should be understood that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device. The electronic device may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0076] Next, the above Figure 1 The electronic device in the embodiment is the execution subject, and the aspect-level sentiment analysis method provided in the embodiment of the present application is exemplarily introduced in combination with the flow chart.

[0077] Specifically, Figure 2 A flow chart of the aspect-level sentiment analysis method provided in this application embodiment is shown in Figure 2 , the method comprising:

[0078] Step S20: Obtain the sentence to be analyzed, and input the sentence to be analyzed into a pre-trained aspect-level sentiment classification model.

[0079] Among them, the aspect-level sentiment classification model includes a text embedding layer, a semantic modeling layer, and a sentiment classification layer.

[0080] Optionally, the sentence to be analyzed can be a sentence carrying aspect information of a specific aspect in the text to be analyzed, consisting of at least one word. The goal of aspect-level sentiment analysis is to predict the emotional tendency of the specific aspect in the sentence, such as positive, neutral, or negative.

[0081] In one example, a text sentence S of a given length n can be represented as the following sequence: S = {w1, w2, ..., w a1 ,w a2 ,…w am ,…,w n-1 ,w n}, its subsequence A={w a1 ,w a2 ,…w am} is the aspect information of length m, consisting of one or more words. The goal of aspect-level sentiment analysis is to predict the sentiment tendency (positive, neutral, negative) of a specific aspect A in sentence S.

[0082] Step S21: Use the serial multi-layer encoding layer in the text embedding layer to extract word vectors for the sentence to be analyzed, and obtain a word vector sequence corresponding to the sentence to be analyzed.

[0083] Among them, the word vector sequence includes the word vectors output by each encoding layer.

[0084] Optionally, the text embedding layer can dynamically encode sentences using a pre-trained model BERT-base, which includes a series of multi-layer Transformer encoding layers to capture sentence context information.

[0085] Optionally, in order to facilitate the text embedding layer to process the sentence to be analyzed, the electronic device may first pre-process the sentence to be analyzed and process it into a format that can be processed by BERT-base.

[0086] In one possible implementation, the format of the sentence to be analyzed can be represented as: [CLS] Sentence [SEP] Aspect [SEP], where [CLS] and [SEP] are special tags used by BERT to classify text and separate text, respectively.

[0087] Optionally, the sentence to be analyzed can be mapped to a word vector H after passing through the text embedding layer i ∈R n×d ,i∈[1,T], where is the output of the i-th Transformer encoding layer, T represents the number of Transformer layers of BERT, n represents the sentence length, d represents the vector dimension, and R represents an n×d-dimensional real vector space. The hidden state vector representing the nth word in the sentence.

[0088] In this embodiment, the pre-trained model BERT-base may have 12 layers, and its output is shown in formula (1).

[0089] H 1 ,H 2 ,…,H T =BERT(X) (1)

[0090] In step S22, the word vector sequence is input into the semantic modeling layer for multi-level modeling and aspect attention analysis to obtain the sentiment feature vector.

[0091] Step S23: Input the sentiment feature vector into the sentiment classification layer for sentiment analysis to obtain the aspect-level sentiment category corresponding to the sentence to be analyzed.

[0092] Optionally, the sentiment classification layer may be a linear layer, which may obtain the probability distribution of sentiment categories through a Softmax function, thereby determining the aspect-level sentiment category corresponding to the sentence to be analyzed.

[0093] In this embodiment, the emotion category probability distribution p can be represented as p=softmax(W p r+b p ), where W p r represents the weight to be trained, b p Characterization bias.

[0094] The aspect-level sentiment analysis method provided by the embodiment of the present application is that the electronic device can input the sentence to be analyzed into a pre-trained aspect and sentiment classification model for processing. The aspect-level sentiment classification model includes a text embedding layer, a semantic modeling layer, and a sentiment classification layer. Among them, the serial multi-layer encoding layer in the text embedding layer can be used to extract word vectors for the sentence to be analyzed, thereby obtaining a word vector sequence corresponding to the sentence to be analyzed, that is, the word vectors output by each encoding layer. Based on this, the semantic modeling layer can be used to perform multi-level modeling and aspect attention analysis on the word vectors output by each encoding layer to obtain a sentiment feature vector, and the sentiment classification layer is used to process the sentiment feature vector to determine the aspect-level sentiment category corresponding to the sentence to be analyzed. In this way, the word vectors output by each encoding layer in the text embedding layer can be fully utilized during the model processing process, thereby solving the problem of semantic overfitting. At the same time, the method can also make full use of low-level semantics and effectively give play to the different advantages of each encoding layer in encoding semantic information, thereby ensuring the full utilization of semantic information and thus improving the accuracy of aspect-level sentiment analysis.

[0095] Optionally, for ease of application, the aspect-level sentiment classification model may be trained in advance.

[0096] In one possible implementation method, the training samples can be input into a pre-built aspect-level sentiment classification model, and the training samples are processed through the text embedding layer, semantic modeling layer, and sentiment classification layer of the model to train the model. The model is then optimized by minimizing the target loss function by cross entropy to obtain a trained aspect-level sentiment classification model.

[0097] In one possible implementation, the objective loss function is represented as: Among them, S represents the number of training samples, C represents the number of emotion categories, Represents the true sentiment label, ∴ represents all trainable parameters, and λ represents the coefficient of the L2 regularization term.

[0098] Optionally, the semantic modeling layer may include a vertical encoding layer, a horizontal multi-granularity layer, and an aspect attention layer. Next, a possible implementation method is provided for how to input the word vector sequence into the semantic modeling layer for multi-level modeling and aspect attention analysis to obtain the sentiment feature vector.

[0099] In this embodiment, the word vector sequence output by the text embedding layer can be input into the vertical encoding layer, and the word vector sequence is bidirectionally modeled using the pyramid attention mechanism to obtain a pyramid graph corresponding to the word vector sequence. The first feature representation corresponding to the root node in the pyramid graph and the target word vector in the word vector sequence are then input into the horizontal multi-granularity layer for multi-scale modeling to obtain a second feature representation. The second feature representation and the target word vector are then input into the aspect attention layer for aspect attention analysis to obtain a sentiment feature vector. The target word vector is the word vector output by the last encoding layer.

[0100] In one possible implementation, Figure 3 For a schematic diagram of the framework of the aspect-level sentiment analysis model, see Figure 3 The aspect-level sentiment analysis model includes a text embedding layer (text embedding module), a vertical encoding layer (vertical encoding layer global semantic module), a horizontal multi-granularity layer (horizontal multi-granularity local semantic module) and an aspect attention layer (aspect attention layer module).

[0101] After processing the sentence to be analyzed into a format that can be processed by BERT-base, the electronic device can input it into the text embedding layer for processing. After processing through multiple layers of Transformer encoding layers, the word vectors output by each layer of Transformer encoding layer are input into the vertical encoding layer for processing, thereby obtaining a pyramid diagram corresponding to the word vector sequence. The first feature representation corresponding to the root node is then input into the horizontal multi-granularity layer for processing, and the output second feature representation is input into the aspect attention layer for processing, thereby obtaining a sentiment feature vector. Finally, sentiment analysis is performed based on the sentiment feature vector to obtain the aspect-level sentiment category corresponding to the sentence to be analyzed.

[0102] Optionally, since the vertical coding layer performs bidirectional modeling of word vectors through the pyramid attention mechanism, the semantic features output by each coding layer can be extracted and fused. Based on this, the first feature representation corresponding to the root node of the pyramid graph actually fuses the dependency information between the sequences of each coding layer and the semantic information captured by each coding layer itself. Therefore, the first feature representation is a feature after aggregating global semantic information.

[0103] It can be understood that the vertical coding layer can fuse the word vectors output by the low-level coding layer in the text embedding layer with the word vectors output by the high-level coding layer through the pyramid attention mechanism, thereby reducing the degree of semantic abstraction and reducing the risk of overfitting. At the same time, the phrases, syntax and other information contained in the low-level coding can enhance semantic understanding, make full use of the information of the low-level coding layer, and improve the utilization rate of semantic information.

[0104] Optionally, the lateral multi-granularity layer can perform multi-scale modeling based on the first feature representation to obtain a second feature representation that can characterize multi-granularity local semantic information. The aspect attention layer can perform processing based on the second feature representation to highlight the sentiment features related to the aspect words, thereby obtaining a sentiment feature vector.

[0105] Next, we provide a possible implementation method for inputting the word vector sequence into the vertical encoding layer, performing bidirectional modeling on the word vector sequence through the pyramid attention mechanism, and obtaining the pyramid graph corresponding to the word vector sequence.

[0106] Specifically, the word vector sequence can be input into the vertical coding layer, and the word vector sequence can be convolved upward layer by layer according to the preset convolution kernel size and the preset step size, and the convolution operation results of each layer can be gated to obtain an initial pyramid diagram. The focus nodes corresponding to each node in the initial pyramid diagram are calculated according to the first preset formula, and the attention value corresponding to each node is calculated according to the focus nodes corresponding to each node and the second preset formula. The feature representation of each node in the initial pyramid diagram is updated according to the attention value of each node to obtain a pyramid diagram corresponding to the word vector sequence.

[0107] Optionally, the preset convolution kernel size and the preset step size can be set according to actual application conditions.

[0108] In this embodiment, the preset convolution kernel size and the preset step size can both be set to C=2. When convolving upward layer by layer, if the number of nodes in a certain layer of convolution sequence is an odd number, the preset convolution kernel size and the preset step size can be set to C=C+1.

[0109] It can be understood that after convolution and gating the word vector sequence layer by layer according to the preset convolution kernel size and preset step size, the initial pyramid diagram obtained is a C-tree, and the receptive field of each layer node to the lower layer node is C, then the receptive field of the root node is the word vector sequence H output by all encoding layers. 1 ,H 2 ,…,H T .

[0110] In this embodiment, the calculation process of the initial pyramid graph is shown in the following formulas (2) and (3).

[0111]

[0112] L j =sigmoid(G j )⊙tanh(G j ) (3)

[0113] Where l represents the maximum number of layers of the initial pyramid graph. The maximum number of layers can be determined according to the preset convolution kernel size, preset step size, and the number of coding layers. For example, if the preset convolution kernel size and preset step size are C=2 and the number of coding layers is 12, then That is to say, the initial pyramid graph includes 4-layer node sequences L1, L2, L3, and L4.

[0114] G j Represents the convolution operation result of the j-1th layer of the pyramid graph, L j Characterize the output result after gated processing. For example, as the j-th node sequence of the pyramid graph, according to the number of coding layers T = 12 and the preset convolution kernel size C = 2, the sequence lengths of L1, L2, L3, and L4 are They are 12, 6, 3, and 1 respectively.

[0115] Functions Sigmoid and tanh form a gate unit, which controls the information transmission ratio and ignores redundant information through gate processing. G Characterize the convolution kernel, b G represents the bias, and ⊙ represents the element-wise product.

[0116] In this embodiment, after obtaining the initial pyramid diagram, the attention nodes corresponding to the nodes in the initial pyramid diagram are determined to establish connections between the nodes inside the pyramid, and the attention mechanism is used to transmit and exchange information, thereby finally obtaining the pyramid diagram.

[0117] In one possible implementation, Figure 4 For a diagram of the pyramid attention mechanism, see Figure 4 , the initial pyramid graph is determined according to each coding layer, and then the connection between nodes in each layer is established.

[0118] In this embodiment, for any node, its corresponding focus nodes include the node itself and the two adjacent nodes before and after it, the parent node of the node and the child node of the node. That is to say, the node connection rule is that the nodes within the sequence are only directly connected to themselves and the two adjacent nodes before and after them, and the nodes between sequences are only directly connected to the parent node and child node. Among them, for this node, the node corresponding to the convolution kernel is the child node, and the new node generated by the convolution is the parent node.

[0119] In this embodiment, the first preset formula is represented by the following formula (4).

[0120]

[0121] in, Represents the attention node corresponding to the m-th node in the j-th layer, Represents the neighboring nodes of the mth node in the jth layer, Represents the child nodes of the mth node in the jth layer, Represents the parent node of the mth node in the jth layer, represents the mth node in the jth layer, A represents the number of neighboring nodes in the same layer, T represents the length of the first-layer node sequence in the pyramid graph, e represents the value range of m, C represents the number of child nodes, and l represents the maximum number of layers in the pyramid graph.

[0122] It can be understood that the neighboring nodes of the m-th node in the j-th layer include the current node.

[0123] In this embodiment, A=3, that is, the number of neighboring nodes in the same layer (including the current node), and T=12 is the length of the first layer node sequence of the initial pyramid graph, that is, the number of BERT-base encoding layers.

[0124] In one example, Figure 5 For a node connection diagram, see Figure 5 , when the node That is, when it is the third node in the second layer, its neighbor nodes in the same layer Child nodes of this node The parent node of this node Based on this, the focus node corresponding to this node

[0125] Optionally, the second preset formula can be represented as formula (5).

[0126]

[0127] Among them, q i The query vector representing the i-th node in the entire pyramid graph, k m The key vector representing the mth node in the jth layer, d k Characterize the feature dimension of the mth node in the jth layer, v m The value vector representing the m-th node in the j-th layer.

[0128] Optionally, after calculating the attention value of each node, the feature representation of each node in the initial pyramid diagram can be updated, as shown in formula (6). After the pyramid attention calculation, the tree structure between levels can summarize the features of different coding layer scales, and the adjacent connections within the level model the sequential dependencies between coding layers. Obviously, the root node of the C-ary tree aggregates the dependency information between BERT coding layer sequences and the semantic information captured by the coding layer itself.

[0129] L lp ,…,L 2p ,L 1p =pyramidal attention(L l ,…,L2,L1) (6)

[0130] Among them, L 1p Characterize the feature representation corresponding to each node in the first layer of the pyramid graph, L 2p Characterize the feature representation corresponding to each node in the second layer of the pyramid graph, L lp Characterize the first feature representation corresponding to the root node of the pyramid graph, L1 characterizes the node sequence of the first layer of the pyramid graph, L2 characterizes the node sequence of the second layer of the pyramid graph, L l Represents the root node sequence of the pyramid graph.

[0131] In this embodiment, Table 1 shows the algorithm flow of the pyramid attention mechanism based on the Transformer encoding layer.

[0132] Table 1 Pyramid Attention Mechanism

[0133]

[0134] It should be noted that in Table 1, p represents the number of pyramid attention layers, that is, the number of times the node is updated using attention.

[0135] In this embodiment, considering that phrases contain richer semantic and emotional information than words, which helps the model understand the text and predict emotional polarity more accurately, the Transformer encoding layer of BERT is mainly composed of a multi-head self-attention mechanism. The advantage of the self-attention mechanism is that it can capture global dependencies, but this global attention method may lead to neglect of local details. For example, when processing text data, the self-attention mechanism may pay more attention to contextual information and long-distance dependencies between words, and is not sensitive enough to subtle features of local areas such as transition words and phrases, thereby affecting the accuracy of sentiment analysis. Therefore, it is also possible to consider processing the first feature representation through a horizontal multi-granularity layer to extract local phrase features of different granularities in the text sequence to enhance the model's ability to utilize multi-granularity local information.

[0136] Specifically, the following provides a possible implementation method for how to input the first feature representation corresponding to the root node in the pyramid diagram and the target word vector in the word vector sequence into the horizontal multi-granularity layer for multi-scale modeling to obtain the second feature representation.

[0137] Specifically, the first feature representation and the target word vector can be input into the horizontal multi-granularity layer, the first feature representation can be processed by multiple parallel one-dimensional convolution layers, and the output results of each one-dimensional convolution layer can be gated to obtain multiple convolution branch results. Then, the multiple convolution branch results can be selectively learned by the Sigmoid function to obtain selective learning results, and the selective learning results and the target word vector can be processed by residual connection and ReLu activation function to obtain the second feature representation.

[0138] Optionally, a multi-scale gated convolution unit is provided in the horizontal multi-granularity layer to extract local phrase features of different granularities in the text sequence to enhance the model's ability to utilize multi-granularity local information.

[0139] In one possible implementation, the number of parallel one-dimensional convolutional layers can be three, see Figure 3 , three parallel one-dimensional convolutions can be set to capture the local interactions between words, and the convolution kernel size k can be set according to the actual application.

[0140] In one possible implementation, the convolution kernel sizes k corresponding to the three parallel one-dimensional convolution layers can be 1, 2, and 4, respectively, and the step sizes can all be 1.

[0141] Optionally, to ensure that the output length is consistent with the input length, the convolution result can be padded with 0 after the convolution.

[0142] In this embodiment, the outputs of convolution branches of different scales can be shown as formulas (7) and (8).

[0143] C k =conv(L lp ,W C )+b C ,k∈{1,2,4} (7)

[0144] S k =sigmoid(C k )⊙tanh(C k ) (8)

[0145] Among them, C k The output of the convolution branch with a convolution kernel size of k contains the local interaction information between words at the k scale, S k Characterize the convolution branch output C k The convolution branch result output after gated tanh unit feature selection, W C Characterize the convolution filter, b C Characterization bias, L lp It represents the first feature representation corresponding to the root node of the pyramid graph, and ⊙ represents the element-by-element product.

[0146] In one example, Figure 6 For a diagram of the convolution operation, see Figure 6 , taking the single-scale convolution with a kernel size of 2 and a stride of 1 as an example, assuming S k ={h1,h2,h3,…,h n-1 ,h n}, where h2 represents the hidden state vector of the second word in the sentence. After the convolution operation, the feature vector Contains the local phrase information of the interaction between word vectors h2 and h3, After the gated tanh unit, the local feature vector after filtering out irrelevant features can be obtained.

[0147] Optionally, in order to achieve selective fusion of features of different granularities, the results of multiple convolution branches can be selectively learned through a Sigmoid function.

[0148] Please continue to see Figure 3 , the weights W can be generated for the convolution branches S1, S2 and S4 respectively through the Sigmoid function 2,4 and W 1,2,4 , and then perform weighted summation step by step to obtain the selective learning result. For details, please refer to the following formulas (9) to (12).

[0149] W 2,4 =Sigmoid(S2+S4) (9)

[0150] S 2,4 =S2×W 2,4 +S4×(1-W 2,4 ) (10)

[0151] W 1,2,4 =Sigmoid(S1+S 2,4 ) (11)

[0152] S 1,2,4 =S1×W 1,2,4 +S 2,4 ×(1-W 1,2,4 ) (12)

[0153] Among them, S 2,4 Represents the weighted result between convolution branches S2 and S4, S 1,2,4 Characterization S 2,4 The weighted result between S1 and S2 is also the selective learning result.

[0154] Optionally, after obtaining the selective learning result, the selective learning result and the target word vector can be processed through residual connection and ReLu activation function. For details, please refer to the following formula (13).

[0155] S out ={S1,S2,…,S n-1 ,S n}=relu(S 1,2,4 +H T ) (13)

[0156] Among them, S out Represents the second feature representation, n represents the length of the text, H T Represents the word vector output by the next encoding layer.

[0157] In this embodiment, the horizontal multi-granularity layer can capture local semantic interactions within different text lengths through convolution operations of different scales, obtain multi-granularity local semantic information, and thus solve the problem of local information loss in the self-attention mechanism.

[0158] Next, we provide a possible implementation method for how to input the second feature representation and the target word vector into the aspect attention layer for aspect attention analysis to obtain the sentiment feature vector.

[0159] The second feature representation and the target word vector are input into the aspect attention layer, the target word vector is processed through the MASK mechanism and pooling operation to obtain the third feature representation, and the second feature representation and the third feature representation are interacted through the third preset formula to obtain the sentiment feature vector.

[0160] In this embodiment, the target word vector can be masked for non-specific aspects through the MASK mechanism, and then a pooling operation is performed on the multi-word aspects to ensure that the aspect lengths are consistent.

[0161] Optionally, the target word vector can be subjected to mask processing and pooling operations using the following formulas (14) to (15).

[0162]

[0163]

[0164] in, Represents the aspect feature vector after non-specific aspect masking, m represents the length of the aspect word, H aspect Representing multi-word aspects, H T Represent the target word vector, Characterize the third feature representation.

[0165] Optionally, the third preset formula can be represented by the following formulas (16) to (18):

[0166]

[0167] Among them, s i Characterize the second feature representation S out The i-th characteristic component in , Represents the third feature representation, r represents the sentiment feature vector, n represents the text length, β i Characterizations i and The cosine similarity of α i Characterization of β i Attention weights normalized by the softmax function.

[0168] Next, in order to further illustrate the effectiveness of the aspect-level sentiment analysis method provided in the embodiment of the present application, the embodiment of the present application evaluated the above-mentioned aspect-level sentiment analysis model through experiments, and compared the aspect-level sentiment analysis model with the model based on static word embedding and the model based on pre-training.

[0169] Specifically, the experimental environment can be Ubuntu 18.04, the GPU can be RTX 3080Ti (12GB), the programming language can be Python 3.8.19, the deep learning framework can be Pytorch 1.12.0, and the development tool can be PyCharm. This experiment can be conducted based on five public datasets, including ACL14 Twitter, SemEval2014 task 4 Rest14 and Lap14, SemEval2015 task 12 Rest15, and SemEval2016 task 5 Rest16. Each sample in this dataset contains a comment sentence, an aspect (consisting of one or more words), and its sentiment polarity (positive, neutral, negative). The sample label distribution is shown in Table 2.

[0170] Table 2 Dataset

[0171]

[0172] The BERT-base word embedding dimension can be set to 768, the batch training sample size can be set to 16, the learning rate and L2 regularization coefficient are both 2e-5, the maximum text length is 100, and the model training is optimized using the Adam optimizer. The parameters of the DeBERTa-based model are consistent with those of the BERT-based model. The main evaluation indicators are accuracy and macro-average F1 value (Macro-F1), as shown in the following formulas (19) to (20).

[0173]

[0174]

[0175] Where T represents the number of correctly predicted samples, and N represents the total number of samples. The F1 score is the harmonic mean of precision and recall. Precision is the proportion of samples predicted as positive by the model that are actually positive, while recall is the proportion of samples correctly predicted as positive by the model out of all samples that are actually positive.

[0176] Next, we compare this aspect-level sentiment analysis model with the following models based on static word embeddings and pre-training: 1. TD-LSTM (Time-Dependent Long Short-Term Memory), which uses two LSTM networks to capture the left-right contextual dependencies of a specific aspect; 2. ATAE-LSTM (Aspect-Term Attention Enhanced LSTM), which adds attention vectors to aspect terms on the basis of LSTM, allowing the model to focus on the key points of the sentence; 3. MemNet (Memory Networks), which focuses on aspect terms in a sentence by fusing multi-hop memory networks and word position attention; 4. IAN (Interactive Attention Network), which uses an interactive attention network to interactively learn the dependencies between context and aspect terms; 5. RAM (Recurrent Attention Model), which proposes an attention-based recursive memory network aspect-level sentiment analysis method that can capture the long-range sentiment characteristics of aspects; and 6. CABASC (Context-Aware Aspect-Based Sentiment Analysis). 7. AOA (Attention-over-Attention), which learns the interaction between aspects and context and focuses on important parts of sentences through an attention-to-attention model. 8. BERT (Bidirectional Encoder Representations from Transformers), which feeds the [CLS] feature vector of the last Transformer encoding layer of BERT into a linear layer for sentiment classification. 9. AEN-BERT, which abandons the traditional recursive mechanism and adopts an attention-based encoder to model the relationship between context and target. 10. BERT-SPC (BERT with Sentence Pair Classification), which fine-tunes BERT by concatenating sentences and aspect words, transforming the target sentiment classification task into a sentence pair classification task.LCF-BERT (Local Context Focus BERT) builds on BERT to model global and local context separately. 12. BERT-SPC-CL (BERT with Sentence Pair Classification and Contrastive Learning) enhances samples through back-translation and synonym replacement, thereby introducing semantic diversity, and then optimizes using a contrastive loss function. 13. SenticGCN (Sentiment-aware Graph Convolutional Network) combines the sentiment knowledge in SenticNet with the dependency tree structure of sentences to construct a sentiment-enhanced graph convolutional network, effectively extracting sentiment dependency information between context words and aspect words. 14. ABSA-DeBERTa (Aspect-Based Sentiment Analysis Decoding-enhanced BERT with disentangled 15. BERT-MATP (BERT with Multi-Aspect Temporal Pooling) proposes a data augmentation method for Masked Aspect Term Prediction (MATP). This method generates diverse aspect terms using a large language model and prompts, and then trains them jointly with the original data. 16. MLEGCN (Multi-Layer Enhanced Graph Convolutional Network) is a framework applicable to all ABSA subtasks. It is a multi-layer graph convolutional network that combines BERT's semantic encoding, bilinear attention, rich linguistic features, and a systematic refinement strategy.

[0177] Through experimental comparison, the aspect-level sentiment analysis model provided in the embodiment of the present application achieved better sentiment classification performance on five data sets. Please see Table 3 for details.

[0178] Table 3 Comparative experimental results of different models (unit: %)

[0179]

[0180]

[0181] It can be seen that compared with non-BERT models, the aspect-level sentiment analysis model provided by the embodiment of the present application achieves comprehensive leading results on all data sets. This is due to BERT's bidirectional encoding layer architecture, which can dynamically represent text and capture bidirectional semantic relationships, thereby obtaining more accurate semantic information. MemNet, AOA, etc. based on static word embeddings cannot dynamically adjust word meanings according to context, and their semantic generalization capabilities are limited, resulting in reduced classification performance.

[0182] Compared with the BERT-based model, the aspect-level sentiment analysis model provided by the embodiment of the present application has significantly improved Accuracy and F1 on multiple data sets. Compared with the SenticGCN model with improved dependency syntax, the aspect-level sentiment analysis model provided by the embodiment of the present application has an average F1 improvement of 0.94% and a maximum improvement of 2.12%, indicating that combining the low-level phrases and syntactic information in BERT with the high-level semantic information can enhance the text feature representation. This is because the fusion of low-level structured features and high-level abstract features reduces the over-parameterization and semantic over-abstraction of high-level encoding, thereby avoiding semantic overfitting caused by relying solely on the final layer features. At the same time, the phrases, syntax and other information contained in the low-level encoding can enhance semantic understanding and make full use of the information of the low-level encoding layer. However, SenticGCN performs better on Twitter and rest15 than the aspect-level sentiment analysis model provided in the embodiment of this application. This is because SenticGCN builds a sentiment knowledge-enhanced graph convolutional network based on BERT, directly models the dependency syntactic tree, and captures the syntactic dependencies between words. This explicit construction of the syntactic dependency graph can better capture syntactic information, while the middle and low-level encoding layers that mainly contain syntactic information capture dependencies by calculating the attention weights between words, but this dependency is implicit and cannot model the syntactic structure as accurately as GCN. Compared with the LCF-BERT model improved by the local context focus mechanism, the F1 of the aspect-level sentiment analysis model provided in the embodiment of this application is improved by 1.5 3%, 3.94%, 1.39%, 6.68%, and 5.97%, indicating that the structure of multiple coding layers and multi-granularity semantics can fully integrate global and local semantics, avoiding the loss of local information in the self-attention mechanism; compared with the AEN-BERT model with improved attention mechanism, the aspect-level sentiment analysis model provided by the embodiment of the present application has leading performance on all data sets, indicating that the cosine attention effectively highlights the emotional features related to aspect words, thereby reducing the information redundancy brought by the structure of multiple coding layers and multi-granularity semantics; BERT-MATP generates enhanced data through a large language model and prompts, and then combines it with the original data for joint training, thereby achieving improved model performance under low data resources, but it is highly dependent on the quality of the generated data, and the computational overhead of the large language model is large.Compared with BERT-MATP, the aspect-level sentiment analysis model provided in the embodiment of the present application has an Acc improvement of 0.26% and an F1 improvement of 0.07% on Rest14. This shows that compared with the method of enriching data semantic information through large language models, the multi-encoding layers and multi-granularity semantic structure of the aspect-level sentiment analysis model provided in the embodiment of the present application can directly capture rich semantic information from existing data with low computational overhead; MLEGCN is a multi-layer graph convolutional network with a relatively complex structure, combining BERT semantic encoding, bilinear attention mechanism, part-of-speech tagging, syntactic dependency, tree structure distance, word pair position information and systematic refinement strategy, and is applicable to all ABSA subtasks. Compared with MLEGCN, the aspect-level sentiment analysis model provided in the embodiment of the present application has an F1 value improvement of 1.10%, 5.60%, 2.71% and 1.67% on Rest14, Lap14, Rest15 and Rest16 for sentiment classification, respectively. This shows that complex downstream structures do not necessarily effectively improve the performance of specific tasks.

[0183] Furthermore, the ABSA-DeBERTa model outperforms the BERT-based model. This is due to the BERT-based model's decoupled attention mechanism, which uses two vectors for each word, representing its content and position, respectively. Attention weights are calculated using two different matrices, leveraging word position information to improve model performance. The traditional BERT model, on the other hand, uses only one vector for each word, ignoring its positional information. Our DeBERTa-based approach achieves superior results to ABSA-DeBERTa, achieving an average F1 improvement of 1.16% and a maximum improvement of 4.2%. This demonstrates that our approach can significantly improve the performance of BERT-based variants in sentiment classification and demonstrates excellent scalability.

[0184] In terms of performance differences across datasets, the aspect-level sentiment analysis model provided by the embodiments of this application shows minimal improvement on Twitter and Rest15, but performs well on Rest14, Lap14, and Rest16. This is presumably due to data characteristics: Twitter text contains a large amount of informal language, which limits the model's ability to rely on local features; Rest15 has a high proportion of complex sentences, requiring the model to simultaneously handle long-range dependencies and multi-faceted sentiment interference; Rest14 and Rest16 texts have standardized structures and clear sentiment expression, enabling the multi-coding layer and multi-granularity semantic structure to efficiently extract aspect-related sentiment clues.

[0185] In order to further verify the effectiveness of the overall structure of the aspect-level sentiment analysis model provided in the embodiment of the present application, different structures in the aspect-level sentiment analysis model provided in the embodiment of the present application can also be removed to carry out ablation experiments. The specific experimental results are shown in Table 4 below.

[0186] Table 4 Ablation experiment results (unit: %)

[0187]

[0188] Please see Table 4, where w / o CA means removing the aspect cosine attention, w / o MGTU means not using the multi-scale gated convolutional unit in the horizontal multi-granularity semantic module, and w / o PA means removing the pyramid attention mechanism.

[0189] As can be seen from Table 4, the overall performance of the aspect-level sentiment analysis model provided by the embodiment of the present application is better than the model after removing different modules. Compared with w / o CA, the aspect-level sentiment analysis model provided by the embodiment of the present application can highlight the sentiment features related to the aspect by calculating the attention between the aspect words and the context, avoiding the information redundancy caused by multiple coding layers and multi-granularity semantics, but the Accuracy of w / o CA on Rest15 is slightly higher than the aspect-level sentiment analysis model provided by the embodiment of the present application. The reason for this is that the model has different sensitivities to the contextual sentiment information of aspect words on different data sets. Specifically, a single comment sample in the Rest15 data set may be a compound sentence composed of multiple sentences, which makes the connection between aspect words and contextual sentiment information not close enough. Compared with w / o MGTU and w / o PA, the results show that the aspect-level sentiment analysis model provided by the embodiment of the present application performs multi-scale and multi-granularity information interaction on the vertical coding layer output sequence and the horizontal text sequence, and can capture richer phrase features, syntactic features and semantic features, so that the model can understand the meaning of the text more accurately.

[0190] Next, we continue to explore the impact of pyramid attention on model performance.

[0191] First, the impact of the number of pyramid attention layers on model performance is analyzed.

[0192] Set the number of attention layers from 1 to 6 for training analysis. Figure 7 For a diagram showing the effect of the number of pyramid attention layers on model performance, see Figure 7 When the number of attention layers is set to 2, the model performance reaches the overall optimal, and as the number of layers increases, the model performance fluctuates and decreases, which indicates that roughly increasing the number of attention layers may weaken the learning ability of the model due to the large increase in parameters, causing overfitting.

[0193] Secondly, the impact of the number of pyramid attention heads on model performance is analyzed.

[0194] The purpose of the multi-head attention mechanism is to enable the model to learn information in parallel in different representation subspaces. Each head represents an independent attention mechanism that can focus on different aspects of the input data, thereby capturing richer features and patterns.

[0195] In order to determine the optimal solution for the number of pyramid attention heads, this experiment set the number of attention heads to 2, 4, 6, 8, 12, and 16 for training analysis. Figure 8 This is a diagram showing the effect of the number of pyramid attention heads on model performance. Figure 8 When the number of attention heads is 4, the comprehensive performance of the aspect-level sentiment analysis model provided by the embodiment of the present application is the best, which means that in this specific task and model architecture, 4 attention heads provide the best balance point, which not only ensures that the model has sufficient representation ability to capture important information in the input, but also does not introduce too many redundant parameters to cause overfitting risks.

[0196] Next, we explore the impact of different convolution scales on model performance.

[0197] To verify the effectiveness of multi-scale gated convolution in extracting local phrase features, comparative experiments can be carried out using convolution kernels of different sizes in the horizontal multi-granularity layer. Figure 9 For a diagram showing the impact of different convolution scales on model performance, see Figure 9 , where Conv(n) means using only one-dimensional convolution with kernel size n to extract local semantic information of n adjacent words, while Conv(a,b,c) means using one-dimensional convolution with kernel sizes a, b, and c to jointly extract local semantic information at multiple scales.

[0198] Depend on Figure 9 It can be seen that the multi-scale joint extraction method represented by Conv(1,2,4) has better results than the single-scale extraction methods represented by Conv(1), Conv(2) and Conv(4). Among them, the accuracy is improved on the other four datasets except the Twitter dataset, especially on the Rest14 and Rest16 datasets, and the F1 value is improved overall. The experimental results verify the effectiveness of extracting phrase features of different granularities in text sequences through multi-scale gated convolution. This multi-scale information fusion helps the model better understand the multi-level features of the input data, thereby improving the model's ability to identify sentiment polarity.

[0199] In summary, the aspect-level sentiment analysis method provided in the embodiment of the present application processes the sentence to be analyzed through the aspect-level sentiment analysis model (the aspect-level sentiment analysis model provided in the embodiment of the present application), extracts word vectors for the sentence to be analyzed through the multi-layer encoding layer passing through the text embedding layer, and fuses the semantic features of each encoding layer through the pyramid attention mechanism, and uses multi-scale gated convolution to capture the local interaction information between words, thereby solving the semantic overfitting, insufficient utilization of low-level semantics and information loss caused by the self-attention mechanism of the BERT model. The t-SNE visualization results show that the sentiment representation clustering ability of the aspect-level sentiment analysis model provided in the embodiment of the present application is better than that of the BERT model. Compared with the classic model LCF-BERT, the aspect-level sentiment analysis model provided in the embodiment of the present application has an F1 improvement of 1.53%, 3.94%, 1.39%, 6.68% and 5.97% on five public data sets respectively. Compared with the latest model SenticGCN, the F1 is improved by 0.94% on average and 2.12% on the largest scale. Compared with ABSA-DeBERTa, the average F1 improvement is 1.16% and the maximum improvement is 4.2%, both achieving excellent sentiment classification results, verifying the effectiveness of integrating BERT's multi-encoding layers and multi-granularity semantics.

[0200] In order to execute the corresponding steps in the above embodiments and various possible methods, an implementation method of an aspect-level sentiment analysis device is given below. Figure 10 , Figure 10 This is a functional module diagram of an aspect-level sentiment analysis device provided in an embodiment of the present application. It should be noted that the basic principles and technical effects of the aspect-level sentiment analysis device provided in this embodiment are the same as those of the aforementioned embodiments. For the sake of brevity, any details not mentioned in this embodiment are referred to the corresponding content in the aforementioned embodiments. The aspect-level sentiment analysis device includes: an acquisition module 200 and a model processing module 210.

[0201] The acquisition module 200 is used to acquire the sentence to be analyzed and input the sentence to be analyzed into a pre-trained aspect-level sentiment classification model; the aspect-level sentiment classification model includes a text embedding layer, a semantic modeling layer and a sentiment classification layer.

[0202] It can be understood that the acquisition module 200 can be used to perform the above step S20.

[0203] The model processing module 210 is used to extract word vectors for the sentence to be analyzed using the serial multi-layer encoding layers in the text embedding layer to obtain a word vector sequence corresponding to the sentence to be analyzed; the word vector sequence includes the word vectors output by each layer of the encoding layer.

[0204] It can be understood that the model processing module 210 can be used to execute the above step S21.

[0205] The model processing module 210 is also used to input the word vector sequence into the semantic modeling layer for multi-level modeling and aspect attention analysis to obtain the sentiment feature vector.

[0206] It can be understood that the model processing module 210 can be used to execute the above step S22.

[0207] The model processing module 210 is further configured to input the sentiment feature vector into the sentiment classification layer for sentiment analysis, and obtain the aspect-level sentiment category corresponding to the sentence to be analyzed.

[0208] It can be understood that the model processing module 210 can be used to execute the above step S23.

[0209] Optionally, the model processing module 210 is also used to input the word vector sequence into the vertical encoding layer, perform bidirectional modeling on the word vector sequence through the pyramid attention mechanism, and obtain a pyramid diagram corresponding to the word vector sequence; input the first feature representation corresponding to the root node in the pyramid diagram and the target word vector in the word vector sequence into the horizontal multi-granularity layer for multi-scale modeling to obtain a second feature representation; the target word vector is the word vector output by the last encoding layer; the second feature representation and the target word vector are input into the aspect attention layer for aspect attention analysis to obtain a sentiment feature vector.

[0210] Optionally, the model processing module 210 is also used to input the word vector sequence into the vertical coding layer, convolve the word vector sequence upward layer by layer according to a preset convolution kernel size and a preset step size, and perform gate processing on the convolution operation results of each layer to obtain an initial pyramid diagram; determine the focus nodes corresponding to each node in the initial pyramid diagram according to a first preset formula; calculate the attention value corresponding to each node according to the focus nodes corresponding to each node and the second preset formula, and update the feature representation of each node in the initial pyramid diagram according to the attention value of each node to obtain a pyramid diagram corresponding to the word vector sequence.

[0211] Optionally, the model processing module 210 is also used to input the first feature representation and the target word vector into a horizontal multi-granularity layer, process the first feature representation through multiple parallel one-dimensional convolution layers, and perform gate processing on the output results of each one-dimensional convolution layer to obtain multiple convolution branch results; selectively learn the multiple convolution branch results through the Sigmoid function to obtain selective learning results, and process the selective learning results and the target word vector through residual connection and ReLu activation function to obtain a second feature representation.

[0212] Optionally, the model processing module 210 is also used to input the second feature representation and the target word vector into the aspect attention layer, process the target word vector through the MASK mechanism and pooling operation to obtain a third feature representation; and interact the second feature representation and the third feature representation through a third preset formula to obtain a sentiment feature vector.

[0213] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory shown in FIG. 1 or the operating system (OS) of the electronic device may be fixed therein and may be Figure 1 Meanwhile, the data and program codes required to execute the above modules may be stored in the memory.

[0214] An embodiment of the present application also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the aspect-level sentiment analysis method provided in the embodiment of the present application can be implemented.

[0215] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the devices, methods, and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a portion of code, and the module, program segment, or a portion of code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0216] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0217] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0218] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for aspect-level sentiment analysis, characterized in that: The method comprises: Obtaining a sentence to be analyzed, and inputting the sentence to be analyzed into a pre-trained aspect-level sentiment classification model; the aspect-level sentiment classification model includes a text embedding layer, a semantic modeling layer, and a sentiment classification layer; Utilizing the serial multi-layer encoding layers in the text embedding layer to extract word vectors from the sentence to be analyzed, and obtaining a word vector sequence corresponding to the sentence to be analyzed; the word vector sequence includes the word vectors output by each layer of the encoding layer; Inputting the word vector sequence into the semantic modeling layer for multi-level modeling and aspect attention analysis to obtain a sentiment feature vector; The sentiment feature vector is input into the sentiment classification layer for sentiment analysis to obtain the aspect-level sentiment category corresponding to the sentence to be analyzed.

2. The method according to claim 1, characterized in that The semantic modeling layer includes a vertical encoding layer, a horizontal multi-granularity layer, and an aspect attention layer; The step of inputting the word vector sequence into the semantic modeling layer for multi-level modeling and aspect attention analysis to obtain a sentiment feature vector includes: Inputting the word vector sequence into the vertical encoding layer, performing bidirectional modeling on the word vector sequence through the pyramid attention mechanism, and obtaining a pyramid graph corresponding to the word vector sequence; Inputting the first feature representation corresponding to the root node in the pyramid graph and the target word vector in the word vector sequence into the horizontal multi-granularity layer for multi-scale modeling to obtain a second feature representation; the target word vector is the word vector output by the last encoding layer; The second feature representation and the target word vector are input into the aspect attention layer for aspect attention analysis to obtain a sentiment feature vector.

3. The method according to claim 2, characterized in that The step of inputting the word vector sequence into the vertical encoding layer, performing bidirectional modeling on the word vector sequence through a pyramid attention mechanism, and obtaining a pyramid graph corresponding to the word vector sequence includes: Inputting the word vector sequence into the vertical coding layer, convolving the word vector sequence layer by layer according to a preset convolution kernel size and a preset step size, and performing gate processing on the convolution operation results of each layer to obtain an initial pyramid map; Determine the focus node corresponding to each of the nodes in the initial pyramid graph according to a first preset formula; According to the focus node corresponding to each of the nodes and the second preset formula, the attention value corresponding to each of the nodes is calculated, and the feature representation of each node in the initial pyramid diagram is updated according to the attention value of each of the nodes to obtain the pyramid diagram corresponding to the word vector sequence.

4. The method according to claim 3, characterized in that The first preset formula is: in, Represents the attention node corresponding to the m-th node in the j-th layer, Represents the neighboring nodes of the mth node in the jth layer, Represents the child nodes of the mth node in the jth layer, Represents the parent node of the mth node in the jth layer, represents the mth node in the jth layer, A represents the number of neighboring nodes in the same layer, T represents the length of the first-layer node sequence in the pyramid graph, e represents the value range of m, and C represents the number of child nodes.

5. The method according to claim 3, characterized in that The second preset formula is: Among them, q i The query vector representing the i-th node in the pyramid graph, k m The key vector representing the mth node in the jth layer, d k Characterize the characteristic dimension of the node, v m The value vector representing the m-th node in the j-th layer.

6. The method according to claim 2, characterized in that The step of inputting the first feature representation corresponding to the root node in the pyramid graph and the target word vector in the word vector sequence into the horizontal multi-granularity layer for multi-scale modeling to obtain a second feature representation includes: Inputting the first feature representation and the target word vector into the horizontal multi-granularity layer, processing the first feature representation through multiple parallel one-dimensional convolutional layers, and performing gate processing on the output results of each of the one-dimensional convolutional layers to obtain multiple convolution branch results; The multiple convolution branch results are selectively learned through the Sigmoid function to obtain a selective learning result, and the selective learning result and the target word vector are processed through the residual connection and the ReLu activation function to obtain the second feature representation.

7. The method according to claim 2, characterized in that The step of inputting the second feature representation and the target word vector into the aspect attention layer for aspect attention analysis to obtain a sentiment feature vector includes: Inputting the second feature representation and the target word vector into the aspect attention layer, processing the target word vector through a mask mechanism and a pooling operation to obtain a third feature representation; The second feature representation and the third feature representation are interacted with each other through a third preset formula to obtain the emotion feature vector.

8. The method according to claim 7, characterized in that The third preset formula is: Among them, s i Characterizes the i-th feature component in the second feature representation, Represents the third feature representation, r represents the emotional feature vector, β i Characterizations i and The cosine similarity of α i Characterization of β i Attention weights normalized by the softmax function.

9. An aspect-level sentiment analysis device, characterized in that: The device comprises: An acquisition module is used to acquire a sentence to be analyzed and input the sentence to be analyzed into a pre-trained aspect-level sentiment classification model; the aspect-level sentiment classification model includes a text embedding layer, a semantic modeling layer, and a sentiment classification layer; A model processing module is used to extract word vectors from the sentence to be analyzed using the serial multi-layer encoding layers in the text embedding layer to obtain a word vector sequence corresponding to the sentence to be analyzed; the word vector sequence includes the word vectors output by each layer of the encoding layer; The model processing module is further configured to input the word vector sequence into the semantic modeling layer for multi-level modeling and aspect attention analysis to obtain a sentiment feature vector; The model processing module is further configured to input the emotion feature vector into the emotion classification layer for emotion analysis, and obtain the aspect-level emotion category corresponding to the sentence to be analyzed.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program that can be executed by the processor, and the processor can execute the computer program to implement the method according to any one of claims 1 to 8.

11. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images