Aspect-level sentiment analysis method fusing multi-granularity features and multi-level fusion network

Abstract

The invention discloses an aspect-level sentiment analysis method fusing multi-granularity features and a multi-level fusion network, and the method comprises the steps: firstly obtaining context features and aspect words of a text through a RoBERTa pre-training model, and converting the context features and aspect words into word vectors for representation; secondly, introducing external knowledge by utilizing a graph convolutional neural network (GCN) in combination with a component dependency tree and a structure tree, and extracting a component structure, a dependency relationship, background knowledge and context emotion features of the text; then, a multi-level fusion filtering module is designed, and the features are filtered and simplified through a cross attention mechanism and an orthogonal projection technology; and finally, predicting the emotion polarity of the text by using a Softmax function. According to the method, the effectiveness of integrating various granularity information features in a complex natural language processing task is shown, the comprehensive task performance of the ABSA model is improved through combination of the various granularity features and an efficient feature fusion technology, and the accuracy of aspect-level sentiment analysis can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to an aspect-level sentiment analysis method integrating multi-granularity features and a multi-level fusion network. Background Art

[0002] Aspect-Based Sentiment Analysis (ABSA) is an important task in Natural Language Processing (NLP). Its core lies in identifying and analyzing sentiment information related to specific aspects in the text, rather than simply classifying the sentiment of the entire text. For example, in a hotel customer review "The appetizers are good but the service is too slow", the reviewer expressed positive sentiment towards "appetizers" and negative sentiment towards "service". Traditional sentiment analysis methods usually only focus on the overall sentiment tendency of the entire review and cannot accurately distinguish the sentiment expressions of different aspects in the text, which is particularly evident in the complex context of multi-faceted sentiment expressions. In order to meet this challenge, ABSA technology came into being, which can accurately identify the sentiment tendencies of multiple aspects in the text and provide technical support for fine-grained sentiment analysis.

[0003] With the development of social media, online reviews and e-commerce platforms, ABSA has been widely used in market research, brand management, customer feedback analysis and other fields. For example, companies can understand consumers' evaluation of specific product features by analyzing user feedback on different aspects of products, and make product improvements or marketing strategy adjustments accordingly. This application demand has led to the rapid development of ABSA technology and the in-depth development of related research in this field. However, the current ABSA task still faces many challenges in practical applications. The most important challenge is how to effectively model aspect words in text and the emotional expressions associated with them. In recent years, the Graph Convolutional Network (GCN) technology based on component trees and dependency trees has performed particularly well in ABSA tasks. The GCN model can effectively transmit the component structure and semantic dependency information in the text, thereby enhancing the accuracy of sentiment analysis. In addition, the introduction of external knowledge has further improved the performance of the model. These technologies provide strong support for ABSA tasks by integrating the structured information of the text and external background knowledge.

[0004] Although some existing research works have made significant progress in some aspects, they still have shortcomings. First, many ABSA models only rely on single-granularity information feature representation or model structure. Although this approach improves performance to some extent, it fails to fully utilize the rich features present in the text. The grammatical structure, semantic features, external knowledge, and contextual information in the text are often interrelated and jointly affect the relationship between aspect words and sentiment words in the text. Existing models often find it difficult to fully integrate these multi-dimensional features, resulting in insufficient performance when dealing with complex natural language texts. Therefore, how to effectively obtain and fuse multi-granular features in ABSA tasks to accurately capture the correlation between aspect words and sentiment words in the text has become a technical problem that needs to be solved urgently.

[0005] In response to the above challenges, many new ABSA methods and systems have emerged in recent years. For example, the patent number CN202011630698.7 discloses an aspect-level sentiment analysis method and system based on deep learning. The invention adopts the interval-feature-based sentiment ternary extraction model SUM-ASTE, and uses a two-layer bidirectional long short-term memory network (BiLSTM) to extract and identify aspect words, opinion words and sentiment polarity in the text. The method first represents each word through a BiLSTM network, and identifies the aspect words in the text based on the representation. Subsequently, the distance information of opinion words and aspect words is fused, and the corresponding sentiment polarity is calculated through the attention mechanism. This method uses a multi-layer deep neural network model to perform high-precision sentiment analysis on specific aspects in the text, effectively improving the accuracy of the analysis. However, this type of model based on long short-term memory networks is easily affected by sequence length when processing long texts, and its ability to model the relationship between aspect words and opinion words is limited when facing complex syntactic structures. In addition, the patent with patent number CN202310284393.2 proposes an aspect-level sentiment analysis method based on the knowledge distillation framework. This method constructs a teacher model and a student model, and uses the grammatical features extracted from the teacher model to guide the extraction of semantic features in the student model, so that the grammatical information of the text can be obtained without using a grammatical structure diagram. The advantage of this method is that it alleviates the noise problem caused by non-grammatical structures in user comments, thereby improving the overall accuracy of the ABSA task. The introduction of knowledge distillation enables the knowledge of complex models to be transferred to simpler models, thereby achieving a good balance between model accuracy and computing resources. However, this method still has limitations when processing sentences without clear aspect words, and the training process of the knowledge distillation framework is highly dependent on the teacher model, and the robustness of the model is poor in some scenarios.

[0006] Although the above research has made significant progress in the field of ABSA, existing methods still have certain limitations in practical applications. First, traditional deep learning methods usually have difficulty in processing grammatical structure, semantic features and contextual information in texts at the same time. Second, existing models have limited use of external knowledge and fail to fully combine the background knowledge in the text to judge sentiment tendency. In addition, many methods have large performance fluctuations when faced with texts of different types and lengths, showing unstable generalization capabilities. Summary of the invention

[0007] In view of the above-mentioned problems, the present invention provides an aspect-level sentiment analysis method that integrates multi-granularity features and multi-level fusion networks, establishes a feature extraction network model based on the combination of dependency tree, structure tree and graph neural network GCN, integrates the component structure information, dependency relationship, context information and external knowledge graph information of aspect-level text sentences, and designs a multi-level fusion network module based on the Cross Attention mechanism and orthogonal projection layer to enhance the performance of ABSA tasks.

[0008] In order to achieve the above object, in the present invention, the aspect-level sentiment analysis method integrating multi-granularity features and multi-level fusion network specifically includes the following steps:

[0009] S1: Text encoding: For the aspect-level text data to be analyzed, the sentence and aspect are combined as input, and the input text is encoded using the pre-trained language model to generate a word vector representation H with context dependency. R , where the word vector performs multi-granularity feature extraction to obtain the component structure relationship, dependency information, external knowledge background and context feature vector representation of the text sentence;

[0010] S2: Multi-granularity feature extraction, specifically including the following steps:

[0011] S2-1: Based on the graph convolutional network, combined with the syntax structure tree and dependency tree, a composite feature extraction framework is constructed:

[0012] Syntactic structure feature extraction: By passing the word vector into the syntactic parser, the syntactic structure tree of the sentence is obtained to form a syntactic structure matrix; each cell of the matrix represents a word or phrase in the text, describing its component structure relationship with other words or phrases, specifically through the adjacency matrix A con ∈R lc×n×n The specific calculation formula is:

[0013]

[0014] Among them, con(m) represents the component structure relationship of the m-level phrase, and m represents the selected l c The level of the phrase in the layer, wi 、w j represents the i-th and j-th words in the text sentence, ph represents the phrase in the text sentence, and u represents the component label associated with the phrase. The component labels specifically include: S, NP, VP; then after multiple layers of graph convolution processing, the node feature H containing the grammatical structure information of the text is obtained con ;

[0015] Dependency feature extraction: Introduce a dependency parser to extract the dependency relationship A between words in a sentence through the probability matrix of the dependency arc dep ∈R n×n Encode and generate a dependency graph to represent the grammatical dependencies between different words in a sentence. The specific calculation formula is:

[0016]

[0017] Among them, link(i,j) indicates that there is a dependency relationship between the i-th and j-th words. Finally, the dependency graph between words is obtained by calculation, which is expressed as H dep ;

[0018] S2-2: External knowledge fusion module: The incoming context feature vector is fused with external knowledge to generate the fused feature representation H kge In addition, in order to retain the original context features to the greatest extent and improve the accuracy of aspect-level sentiment analysis, an independent channel is set up to directly retain the context features H generated by the pre-trained language model. R ;

[0019] Through steps S2-1 and S2-2, the output of multi-granularity feature extraction specifically includes: con , H dep , H kge and H R , set up a multi-layer perceptron layer, map and transform the features through the linear layer and ReLU activation function, the specific output features include: node output feature F con , dependency graph output feature F dep , fusion output feature F kge , context output feature F R ;

[0020] S3: Multi-level fusion, including the following steps:

[0021] S3-1: Cross-attention mechanism: Input the output feature F of step S2 con 、F dep 、F R , grouping them into F con +F R and F dep+F R , CrossAttention is used to combine the two to obtain the context-structure feature representation W1 and the context-dependency feature representation W2 in the text, thereby introducing the synergistic interaction between various granularities of the text;

[0022] S3-2: Orthogonal projection layer: input the output feature F of step S2 con 、F dep 、F R , the text sentence structure feature F con Projected to context feature F R The specific calculation formula is:

[0023]

[0024] Among them, Proj() is the projection function, which projects in the orthogonal direction of the feature to obtain a more refined feature F con* ; Similarly, the dependency feature F dep Projected to context feature F R On the top, we get the refined feature F dep* ;

[0025] S3-3: Multi-level fusion: For the features after the cross attention and orthogonal projection layers, the weighted multiplication fusion strategy is used to perform weighted fusion on each feature to effectively combine the synergy of each feature. At the same time, the Dropout layer is introduced to prevent overfitting, and the fused features are normalized through the multi-feature scalable co-attention fusion module to accelerate training and improve the generalization ability of the model;

[0026] S4: Sentiment polarity judgment: Based on the fusion features, the sentiment probability distribution of the final text sentence is calculated through a linear layer of a Softmax function.

[0027] As a technical preferred solution of the present invention, in step S1, the pre-trained language model is a RoBERTa model, which is used to generate word vector representations while performing multi-granularity feature extraction on the word vectors, including component structure relationships, dependency information, external knowledge background, and context feature vector representations of text sentences; this process ensures comprehensive semantic capture of the input text and accuracy of sentiment analysis.

[0028] As a technical preferred solution of the present invention, in step S2-1, in the combination of the graph convolutional network GCN and the syntactic structure tree, the input word vector is converted into a syntactic structure matrix by a syntactic parser to represent the component structure relationship in the sentence; each cell in the syntactic structure matrix represents a word or phrase in the input text, and is calculated through the corresponding adjacency matrix, which is defined by component labels and phrase levels, wherein the component labels specifically include: S, NP, VP.

[0029] As a technical preferred solution of the present invention, the multi-level fusion in step S3-3 specifically includes the following steps:

[0030] T1: Combine the context-structure feature representation W1, the context-dependency feature representation W2 and the projected features with the external knowledge feature F kge , perform feature fusion;

[0031] T2: Integrate the correlation between multiple features through element multiplication;

[0032] T3: Use the Dropout layer to prevent the model from overfitting;

[0033] T4: The multi-feature scalable co-attention fusion method MAMN is used to normalize the fused features to accelerate training and improve the generalization ability of the model;

[0034] T5: The final output fused features, the formula is as follows:

[0035]

[0036] in, represents the learnable weight parameter, Norm represents the normalization layer, and Z i Represents the output of the i-th layer module.

[0037] As a technical preferred solution of the present invention, the multi-level fusion includes feature simplification, weight multiplication fusion, and normalization operation, and the overall module is repeatedly calculated and fused. Each time the output Z in the module is i After the Mean-Pooling operation, the average eigenvalue is calculated, and finally the feature representation R containing 5 different feature fusions is obtained. The specific calculation formula is as follows:

[0038] R=Mean(Z 1 ,Z 2 ,...,Z m )

[0039] Among them, R represents the output of the final feature, m represents the number of module layers, and Mean() represents the average pooling function.

[0040] As a technical preferred solution of the present invention, the emotion probability distribution calculation output in step S4 is as follows:

[0041] P = Softmax(W R R+b R )

[0042] Wherein, R represents the comprehensive feature representation obtained from the multi-level fusion filtering module, which is the result of weighted fusion of the features after processing at each layer in step S3; W R is the weight matrix, which is used for linear transformation in the sentiment polarity judgment step; b R is a bias term used to adjust the activation value of the output layer

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The present invention proposes a multi-granularity feature extraction module, which aims to capture the responsible interactions between features of different granularities by integrating four granularity features, namely component structure, dependency, contextual features and external knowledge, so as to achieve more accurate judgment of the sentiment polarity of aspect words.

[0045] Taking into account the complexity and diversity of the multi-granularity features of the input, the present invention designs a multi-level fusion module, which realizes mutual learning between various granularities through the use of multi-layer CrossAttention mechanism and orthogonal projection technology, thereby obtaining more refined syntactic and semantic features.

[0046] The aspect-level sentiment analysis method proposed in the present invention demonstrates excellent performance. Compared with the existing baseline model, the accuracy rates on the aspect-level public datasets Restuarant and Laptop datasets reached 88.05% and 81.43% respectively. It also achieved a significant improvement in the comprehensive performance indicator F1 value, surpassing most of the more representative ABSA methods at this stage.

[0047] In summary, the present invention aims at the problem of aspect-level sentiment analysis and proposes an aspect-level sentiment analysis method that integrates multi-granularity features and a multi-level fusion network. This method not only demonstrates the effectiveness of integrating multiple granularity information features in complex natural language processing tasks, but also provides a train of thought and method for future research through the design of a multi-level fusion filtering network, that is, to improve the comprehensive task performance of the ABSA model by combining multiple granularity features with efficient feature fusion technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is an overall architecture diagram of the present invention that integrates multi-granularity features and a multi-level fusion network model;

[0049] Figure 2 It is an example diagram of the dependency tree and component structure tree of the present invention;

[0050] Figure 3 This is an example diagram of the orthogonal projection technology used in the multi-level fusion module of the present invention. DETAILED DESCRIPTION

[0051] The present invention is further described below in conjunction with the accompanying drawings and examples. However, the present invention can be implemented in many different ways and should not be construed as being limited to the embodiments shown; on the contrary, these embodiments provide those skilled in the art with implementation methods that meet applicable legal requirements.

[0052] Example 1: Figure 1 As shown, in the present invention, an aspect-level sentiment analysis model integrating multi-granularity features and a multi-level fusion network effectively integrates component structure features, dependency information, contextual features and external knowledge, and designs a multi-level fusion network module to deeply capture the mutual synergy between granular features. One of the innovations of this method is that the Cross Attention mechanism and orthogonal projection technology are used in the feature fusion filtering stage. The full combination of these two technologies effectively utilizes the comprehensive potential and synergy between various granular features. It not only starts from the context-structure features and context-dependency features of the text sentences, but also effectively reduces the computational burden brought by the fusion of multi-granularity features through projection transformation between features, while ensuring the improvement of model performance and reducing the redundancy of data to a certain extent.

[0053] The aspect-level sentiment analysis method integrating multi-granularity features and multi-level fusion networks provided by the present invention comprises the following steps:

[0054] S1: Text encoding: For the aspect-level text data to be analyzed, the sentence and aspect are combined as input, and the input text is encoded using the pre-trained language model to generate a word vector representation H with context dependency. R , where the word vector performs multi-granularity feature extraction to obtain the component structure relationship, dependency information, external knowledge background and context feature vector representation of the text sentence;

[0055] S2: Multi-granularity feature extraction, specifically including the following steps:

[0056] S2-1: Based on the graph convolutional network, combined with the syntax structure tree and dependency tree, a composite feature extraction framework is constructed:

[0057] Syntactic structure feature extraction: By passing the word vector into the syntactic parser, the syntactic structure tree of the sentence is obtained to form a syntactic structure matrix; each cell of the matrix represents a word or phrase in the text, describing its component structure relationship with other words or phrases, specifically through the adjacency matrix A con ∈R lc×n×n The specific calculation formula is:

[0058]

[0059] Among them, con(m) represents the component structure relationship of the m-level phrase, and m represents the selected l c The level of the phrase in the layer, w i 、w j represents the i-th and j-th words in the text sentence, ph represents the phrase in the text sentence, and u represents the component label associated with the phrase. The component labels specifically include: S, NP, VP; then after multiple layers of graph convolution processing, the node feature H containing the grammatical structure information of the text is obtained con ;

[0060] Dependency feature extraction: Introduce a dependency parser to extract the dependency relationship A between words in a sentence through the probability matrix of the dependency arc dep ∈R n×n Encode and generate a dependency graph to represent the grammatical dependencies between different words in a sentence. The specific calculation formula is:

[0061]

[0062] Among them, link(i,j) indicates that there is a dependency relationship between the i-th and j-th words. Finally, the dependency graph between words is obtained by calculation, which is expressed as H dep ;

[0063] S2-2: External knowledge fusion module: The incoming context feature vector is fused with external knowledge to generate the fused feature representation H kge In addition, in order to retain the original context features to the greatest extent and improve the accuracy of aspect-level sentiment analysis, an independent channel is set up to directly retain the context features H generated by the pre-trained language model. R ;

[0064] Through steps S2-1 and S2-2, the output of multi-granularity feature extraction specifically includes: con , H dep , H kge and H R , set up a multi-layer perceptron layer, map and transform the features through the linear layer and ReLU activation function, the specific output features include: node output feature F con, dependency graph output feature F dep , fusion output feature F kge , context output feature F R ;

[0065] S3: Multi-level fusion, including the following steps:

[0066] S3-1: Cross-attention mechanism: Input the output feature F of step S2 con 、F dep 、F R , grouping them into F con +F R and F dep +F R , CrossAttention is used to combine the two to obtain the context-structure feature representation W1 and the context-dependency feature representation W2 in the text, thereby introducing the synergistic interaction between various granularities of the text;

[0067] S3-2: Orthogonal projection layer: input the output feature F of step S2 con 、F dep 、F R , the text sentence structure feature F con Projected to context feature F R The specific calculation formula is:

[0068]

[0069] Among them, Proj() is the projection function, which projects in the orthogonal direction of the feature to obtain a more refined feature F con* ; Similarly, the dependency feature F dep Projected to context feature F R On the top, we get the refined feature F dep* ;

[0070] S3-3: Multi-level fusion: For the features after the cross attention and orthogonal projection layers, the weighted multiplication fusion strategy is used to perform weighted fusion on each feature to effectively combine the synergy of each feature. At the same time, the Dropout layer is introduced to prevent overfitting, and the fused features are normalized through the multi-feature scalable co-attention fusion module to accelerate training and improve the generalization ability of the model;

[0071] S4: Sentiment polarity judgment: Based on the fusion features, the sentiment probability distribution of the final text sentence is calculated through a linear layer of a Softmax function.

[0072] Example 2: This example adopts the ABSA task to introduce the implementation process and experimental results of the invention in detail. The specific analysis method flow is as follows:

[0073] S1: Text encoding. In the ABSA task, given a sentence consisting of n words: S = {w1,w2,w3,···,w n}, the sentence includes an aspect word sequence A of length m = {A1, A2, A3, ···, A m}, and its corresponding sentiment polarity C a Among them, w i represents the i-th word in the context, A i represents the i-th aspect word, aspect A is a subsequence of sentence S, C a ∈{Positive, Neutral, Negative}. In order to obtain a more comprehensive context feature representation and considering the shortcomings of the traditional BERT model, the present invention adopts the RoBERTa model to map each word to a low-dimensional real-valued vector space and construct a word vector composed of low-dimensional real number vectors.

[0074] In the RoBERTa encoder, in order to facilitate the input of sentence and aspect word information, a sentence-aspect pair is constructed as input, which is specifically expressed as: x = ([CLS] S[SEP] A[SEP]). After being processed by RoBERTa, it is expressed as:

[0075]

[0076] Where d represents the dimension of RoBERTa embedding, Represents the context representation corresponding to the i-th word. Through this method, the contextual information of the text can be deeply mined, providing a solid and powerful foundation for analyzing the complex relationship between aspect words and opinion words.

[0077] S2: Multi-granularity feature extraction. The existing aspect-level sentiment analysis model only extracts one or several features of the grammatical, semantic and other features of the text sentence, and there is no relatively complete multi-granularity feature extraction module. In recent years, the combination of graph convolutional neural network GCN and dependency tree and structure tree has achieved good results. The introduction of external knowledge and contextual features has also effectively improved the performance of ABSA tasks. The present invention comprehensively analyzes the complex relationship between aspect words and opinion words from the perspective of four granularity features: grammatical component structure of text sentences, semantic dependency information, external knowledge and contextual features.

[0078] S2-1: Based on the graph convolutional network, combined with the syntax structure tree and dependency tree, a composite feature extraction framework is constructed:

[0079] Syntactic structure features, the features H output from the text encoding module RThe input to the component parser is converted into a syntactic structure matrix, also called a syntactic analysis tree. In this matrix, each cell represents a word or phrase in the input sentence and describes the syntactic component relationship between them. Specifically, it is represented by the adjacency matrix A con ∈R lc×n×n , specifically defined as follows:

[0080]

[0081] Among them, con(m) represents the component structure relationship of the m-level phrase, and m represents the selected l c The level of the phrase in the layer, w i 、w j Represents the i-th and j-th words in the text sentence, ph represents the phrase in the text sentence, and u represents the component label associated with the phrase, such as S, NP, VP, etc. The output hidden representation is then generated by calculation The specific formula is as follows:

[0082]

[0083] Among them, W l represents the weight matrix, b l represents the bias term, and σ is the activation function.

[0084] Dependency features, the combination of graph convolutional neural network GCN and dependency tree, through the context feature vector H R As input, the probability matrix of all dependency arcs in the dependency parser is used to encode the grammatical information, which is specifically represented as A dep ∈R n×n , its specific definition is as follows:

[0085]

[0086] Among them, link(i,j) indicates that there is a dependency relationship between the i-th and j-th words. Finally, the dependency graph between words is obtained by calculation, which is expressed as The calculation formula is as follows:

[0087]

[0088] Among them, W l represents the weight matrix, b l represents the bias term, and σ is the activation function.

[0089] S2-2: External knowledge fusion, targeting the context feature information H output by the text encoding module RoBERTa model R , and integrate it with the external knowledge structure, and obtain the feature H containing relevant knowledge background through calculationkge , specifically expressed as:

[0090] Contextual features, in order to make full use of the language representation learned by the RoBERTa pre-training model and retain the feature representation of the original context to the greatest extent, so as to improve the performance of the ABSA task, a separate channel is designed to directly retain the features generated by the text encoding module without any processing. The specific expression is:

[0091]

[0092] Where d represents the dimension of RoBERTa embedding, Represents the context representation corresponding to the i-th word.

[0093] The feature linear transformation layer is used to extract the four features H con , H dep , H kge , H R , introduce the MLP module to perform deep mapping and transformation of features, so as to better capture the relationship between features and enhance the model's ability to express features. In the hidden layer, the linear layer and ReLU function are used for transformation, and after the Dropout operation, some elements in the input vector are randomly set to 0 during the training process to prevent overfitting. Finally, in the output layer, the output vector of the hidden layer is linearly transformed again to obtain the final feature vector output. The specific output is expressed as: F con 、F dep 、F kge 、F R .

[0094] S3: Multi-level fusion, after the multi-granularity feature extraction module outputs four features F con 、F dep 、F kge 、F R, which describes the feature information of the text from different angles. However, the current processing and fusion methods for multiple granular features are relatively simple, which does not fully utilize the characteristics and related advantages of each feature, and easily leads to high data complexity and redundancy. To address this problem, a multi-level feature fusion module is introduced, which extracts more representative features through attention mechanism and orthogonal projection technology, while reducing the redundancy of features to a certain extent, thereby reducing the computational cost, and finally performing fusion. Specifically, it includes the following three modules: CrossAttention module, which realizes the fusion of features through cross-attention and obtains the feature representation of context-structure features and context-dependency; orthogonal projection layer, which reduces the feature dimension to a more appropriate level through orthogonal projection technology, thereby reducing the complexity and computational cost of the model; feature fusion, which is converted into the final sentiment polarity output through a series of fusion operations such as vector multiplication and regularization.

[0095] S3-1: Cross-attention mechanism for multi-granularity features F con 、F dep 、F kge 、F R , considering F kge The features already contain rich semantic and syntactic information, so kge The features are not processed in any way and are directly passed to the fusion layer. con 、F dep 、F R Considering the characteristics of different features, two Cross Attention modules are designed to process F con 、F R and F dep 、F R Features, feature interaction, and the interactive attention mechanism is used to combine the two, so that the model can better utilize the relationship between the two representations. con 、F R Afterwards, F R Q, F as the attention layer con As K and V, the context-structure feature representation W1 is obtained by calculation, and the calculation formula is as follows:

[0096]

[0097] Similarly, for the input feature F dep 、F R , the context-dependent feature representation W2 is obtained by calculation, and the calculation formula is as follows:

[0098]

[0099] Among them, dk represents the kth vector dimension.

[0100] S3-2: Orthogonal projection layer. Given the diversity and complexity of multi-granular features, the computational cost of the model is increased to a certain extent, and data redundancy is easily generated. To address this problem, the orthogonal projection technology is introduced to achieve multiple benefits by performing a two-dimensional transformation on the multi-granular feature vector space. First, the correlation between features is reduced, making the model more robust and less susceptible to excessive influence of a certain type of feature; second, by enhancing the expressive power of the model, the orthogonal projection technology can capture the emotional information in the multi-granular features more finely and improve the performance of the ABSA task; finally, the use of orthogonal projection technology helps to simplify the complexity of the model, especially for the particularity of multi-granular features, which can effectively improve computational efficiency and make the model more suitable for large-scale data sets.

[0101] First, the sentence structure feature F con Projected to context feature F R On, as shown below:

[0102]

[0103] Among them, Proj() is the projection function.

[0104] Secondly, in feature F con Projecting in the orthogonal direction of , we get a finer feature as shown below:

[0105]

[0106] Similarly, the dependency feature F dep Projection to F R The specific calculation is as follows:

[0107]

[0108]

[0109] S3-3: Multi-level fusion, for input features W1, W2, and F kge The five features are first integrated through the element-wise multiplication layer to integrate the correlation between multiple features, and the Dropout layer is used to prevent overfitting. The output results are then normalized to accelerate training and improve the generalization ability of the model. The feature fusion layer is integrated into a module to perform repeated calculations and fusion. Finally, Mean-Pooling is performed to calculate the average feature output to obtain the feature R with five different feature fusions, as shown below:

[0110]

[0111] R=Mean(Z 1 ,Z 2 ,...,Z m )

[0112] Among them, all represents the learnable weight parameter, Norm represents the normalization layer, and Z i Represents the output of the i-th layer module, R represents the output of the final feature, m represents the number of module layers, and Mean() represents the average pooling function.

[0113] S4: Sentiment polarity judgment, using the final output feature R obtained by the multi-level feature fusion module, and the final sentiment classification prediction probability is obtained through the Softmax function, as shown below:

[0114] P = Softmax(W R R+b R )

[0115] Among them, W R is the weight matrix, b R is the bias term.

[0116] In order to prove the effectiveness of the method proposed in this invention, experiments are conducted on two public datasets, Laptop and Restaurant. The datasets used are from SemEval2014. The detailed information of the datasets is shown in Table 1. Accuracy and F1 value are used as method evaluation indicators.

[0117] Table 1: Dataset details

[0118]

[0119] Table 2: Results of the dataset on different models

[0120]

[0121] Table 2 accurately summarizes the key experimental results. The proposed method, i.e., the aspect-level sentiment analysis method integrating multi-granularity features and multi-level fusion networks, has achieved the best performance level on the Laptop and Restuarant datasets compared with other comparison models, which well proves the feasibility and effectiveness of the proposed method in integrating multi-granularity features and feature fusion filtering. By comparison, it is found that the present invention also achieves good results in the text encoding stage compared with the traditional BERT by using the RoBERTa model.

[0122] However, if the model only relies on a certain granularity feature and ignores the importance of other sentiment information, it will often make sentiment judgment errors, especially in informal or complex text sentences with large amounts of information. The present invention, starting from multiple granularity features, can provide a more comprehensive understanding of text sentences and achieve better performance. This also indirectly emphasizes the importance of combining multiple granularity features to grasp the sentiment polarity of the overall text.

[0123] The experimental results clearly show that by integrating information of four different granularities, namely, component structure features, dependency information, contextual context and external knowledge, and combining the Cross Attention mechanism and orthogonal projection technology, deeper feature simplification and fusion are performed, which improves the performance of the overall ABSA task while reducing data redundancy to a certain extent. This is not only innovative in theory, but also shows significant performance improvement in specific practical applications, providing a strong reference and inspiration for further improving the research on ABSA tasks in the future.

[0124] The above embodiments only express the implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention.

Claims

1. An aspect-level sentiment analysis method integrating multi-granularity features and multi-level fusion networks, characterized by: The following steps are involved: S1: Text encoding: For the aspect-level text data to be analyzed, the sentence and aspect are combined as input, and the input text is encoded using the pre-trained language model to generate a word vector representation H with context dependency. R , where the word vector performs multi-granularity feature extraction to obtain the component structure relationship, dependency information, external knowledge background and context feature vector representation of the text sentence; S2: Multi-granularity feature extraction, specifically including the following steps: S2-1: Based on the graph convolutional network, combined with the syntax structure tree and dependency tree, a composite feature extraction framework is constructed: Syntactic structure feature extraction: By passing the word vector into the syntactic parser, the syntactic structure tree of the sentence is obtained to form a syntactic structure matrix; each cell of the matrix represents a word or phrase in the text, describing its component structure relationship with other words or phrases, specifically through the adjacency matrix A con ∈R lc×n×n The specific calculation formula is: Among them, con(m) represents the component structure relationship of the m-level phrase, and m represents the selected l c The level of the phrase in the layer, w i 、w j represents the i-th and j-th words in the text sentence, ph represents the phrase in the text sentence, and u represents the component label associated with the phrase. The component labels specifically include: S, NP, VP; then after multiple layers of graph convolution processing, the node feature H containing the grammatical structure information of the text is obtained con ; Dependency feature extraction: Introduce a dependency parser to extract the dependency relationship A between words in a sentence through the probability matrix of the dependency arc dep ∈R n×n Encode and generate a dependency graph to represent the grammatical dependencies between different words in a sentence. The specific calculation formula is: Among them, link(i,j) indicates that there is a dependency relationship between the i-th and j-th words. Finally, the dependency graph between words is obtained by calculation, which is expressed as H dep ; S2-2: External knowledge fusion module: The incoming context feature vector is fused with external knowledge to generate the fused feature representation H kge In addition, in order to retain the original context features to the greatest extent and improve the accuracy of aspect-level sentiment analysis, an independent channel is set up to directly retain the context features H generated by the pre-trained language model. R ; Through steps S2-1 and S2-2, the output of multi-granularity feature extraction specifically includes: con , H dep , H kge and H R , set up a multi-layer perceptron layer, map and transform the features through the linear layer and ReLU activation function, the specific output features include: node output feature F con , dependency graph output feature F dep , fusion output feature F kge , context output feature F R ; S3: Multi-level fusion, including the following steps: S3-1: Cross-attention mechanism: Input the output feature F of step S2 con 、F dep 、F R , grouping them into F con +F R and F dep +F R , CrossAttention is used to combine the two to obtain the context-structure feature representation W1 and the context-dependency feature representation W2 in the text, thereby introducing the synergistic interaction between various granularities of the text; S3-2: Orthogonal projection layer: input the output feature F of step S2 con 、F dep 、F R , the text sentence structure feature F con Projected to context feature F R The specific calculation formula is: Among them, Proj() is a projection function, which projects in the orthogonal direction of the feature to obtain a more refined feature Similarly, the dependency feature F dep Projected to context feature F R The refined features are obtained S3-3: Multi-level fusion: For the features after the cross attention and orthogonal projection layers, the weighted multiplication fusion strategy is used to perform weighted fusion on each feature to effectively combine the synergy of each feature. At the same time, the Dropout layer is introduced to prevent overfitting, and the fused features are normalized through the multi-feature scalable co-attention fusion module to accelerate training and improve the generalization ability of the model; S4: Sentiment polarity judgment: Based on the fusion features, the sentiment probability distribution of the final text sentence is calculated through a linear layer of a Softmax function.

2. The aspect-level sentiment analysis method integrating multi-granularity features and multi-level fusion networks according to claim 1, characterized in that: In step S1, the pre-trained language model is a RoBERTa model, which is used to generate word vector representations while performing multi-granular feature extraction on the word vectors, including component structure relationships, dependency information, external knowledge background, and context feature vector representations of text sentences; This process ensures comprehensive semantic capture of the input text and accuracy of sentiment analysis.

3. The aspect-level sentiment analysis method integrating multi-granularity features and multi-level fusion networks according to claim 1, characterized in that: In step S2-1, in the combination of the graph convolutional network GCN and the syntactic structure tree, the input word vector is converted into a syntactic structure matrix by a syntactic parser to represent the component structure relationship in the sentence; each cell in the syntactic structure matrix represents a word or phrase in the input text, and is calculated through the corresponding adjacency matrix, which is defined by the component label and the phrase level, where the component labels specifically include: S, NP, VP.

4. The aspect-level sentiment analysis method integrating multi-granularity features and multi-level fusion networks according to claim 1, characterized in that: The multi-level fusion in step S3-3 specifically includes the following steps: T1: Combine the context-structure feature representation W1, the context-dependency feature representation W2 and the projected features with the external knowledge feature F kge , perform feature fusion; T2: Integrate the correlation between multiple features through element multiplication; T3: Use the Dropout layer to prevent the model from overfitting; T4: The multi-feature scalable co-attention fusion method MAMN is used to normalize the fused features to accelerate training and improve the generalization ability of the model; T5: The final output fused features, the formula is as follows: in, represents the learnable weight parameter, Norm represents the normalization layer, and Z i Represents the output of the i-th layer module.

5. The aspect-level sentiment analysis method integrating multi-granularity features and multi-level fusion networks according to claim 1, characterized in that: The multi-level fusion includes feature simplification, weight multiplication fusion, and normalization operations. The whole module is repeatedly calculated and fused. The output Z of each module is i After the Mean-Pooling operation, the average eigenvalue is calculated, and finally the feature representation R containing 5 different feature fusions is obtained. The specific calculation formula is as follows: R=Mean(Z 1 ,Z 2 ,...,Z m ) Among them, R represents the output of the final feature, m represents the number of module layers, and Mean() represents the average pooling function.

6. The aspect-level sentiment analysis method integrating multi-granularity features and multi-level fusion networks according to claim 1, characterized in that: The emotion probability distribution output in step S4 is calculated as follows: P=Softmax(W R R+b R ) Wherein, R represents the comprehensive feature representation obtained from the multi-level fusion filtering module, which is the result of weighted fusion of the features after processing at each layer in step S3; W R is the weight matrix, which is used for linear transformation in the sentiment polarity judgment step; b R is a bias term used to adjust the activation value of the output layer.

Images