An aspect-level sentiment analysis method based on multi-grammar and multi-frequency

Abstract

The application provides an aspect-level sentiment analysis method based on multi-grammar and multi-frequency, and belongs to the technical field of aspect-level sentiment analysis; in order to solve the technical problems that the sentiment judgment of the current aspect-level sentiment analysis is not accurate enough, and the dynamic fusion is not performed, leading to the performance decline of a model, the technical scheme is that: a full-chain graph convolutional neural network is used to capture the grammar information and the dependency relationship of a comment, then the comment is analyzed in combination with semantic role labeling and abstract semantic representation, and multi-frequency propagation is used to capture the semantic correlation relationship of each word in the text, a double-channel graph neural network is constructed, then an alignment operation is used to convert the full-chain graph encoding and the semantic graph encoding to a unified feature space, and finally a dynamic fusion mechanism is used to obtain comprehensive sentence expression, so that the performance of aspect-level sentiment analysis is improved; the application is applied to aspect-level sentiment analysis.

Description

Technical Field

[0001] This invention provides an aspect-level sentiment analysis method based on multiple syntaxes and multiple frequencies, belonging to the field of aspect-level sentiment analysis technology. Background Technology

[0002] The internet has become an integral part of all sectors of society. Every day, social networks and online portals generate a vast amount of emotionally charged data, covering multiple fields such as books, people, hotels, and restaurants, containing enormous commercial and social value. Research shows that consumers are more inclined to trust reviews and ratings from strangers before purchasing products or services, and subconsciously, they are more willing to buy goods or services with a large number of positive reviews. Therefore, it is particularly important to use sentiment analysis technology to extract the value of sentiment terms and emotional polarities from large amounts of review data.

[0003] Aspect-level sentiment analysis (also known as fine-grained sentiment analysis) is a sophisticated and detailed sentiment analysis task. Its core objective is to accurately identify aspect terms in comment text and classify their sentiment tendencies. Traditional aspect-level sentiment analysis methods mainly rely on feature engineering to train traditional machine learning classifiers. The performance of these methods heavily depends on handcrafted features, which consumes a significant amount of human and material resources. In recent years, with the rapid development of neural network technology, deep neural networks have achieved remarkable results in many application fields. Against this backdrop, research on aspect-level sentiment analysis has also undergone a shift from traditional methods relying on feature engineering to deep learning methods. Currently, deep learning-based models can be mainly summarized as follows: methods based on recurrent neural networks (RNNs), methods based on convolutional neural networks (CNNs), methods incorporating attention mechanisms, methods utilizing pre-trained models, and methods based on graph neural networks. Each of these methods has unique advantages and has shown great potential in the field of aspect-level sentiment analysis.

[0004] The methods described above have been widely used in aspect-level sentiment analysis, but most works have not considered incorporating multi-grammatical information, such as dependency structures, semantic role labeling, and abstract semantic representations, into aspect-level sentiment analysis tasks. Only a few works have considered dependency tree and component tree structural information, and these works have the following shortcomings:

[0005] (1) It only focuses on single grammatical structure information and does not consider the fusion effect of multiple grammatical representations. The impact of different label information in semantic role labeling and abstract semantic representation on aspect-level sentiment analysis performance is different. Semantic role labeling identifies the predicates and arguments (such as agent, patient, time, etc.) in the sentence to clarify the semantic role of each component, which helps the model to more accurately locate the dependency relationship between aspect words and sentiment words and reduce misjudgment caused by syntactic ambiguity. Abstract semantic representation integrates the core semantic relationship of the sentence (such as action, state, causal relationship) with graph structure to capture deep semantics beyond syntactic structure. This global semantic perspective enhances contextual relevance, especially when dealing with implicit sentiment or complex sentence structure, and can extract key semantic features more stably.

[0006] (2) The guidance role of word frequency information is ignored, resulting in inaccurate sentiment judgment. The combination of words with different frequencies can form unique semantic patterns, which provide an important basis for judging sentiment polarity. If this dimension is ignored, the model may make misjudgments when dealing with complex emotional expressions or polysemous words, weakening its ability to capture contextual sensitivity and ultimately affecting the accuracy of the analysis results.

[0007] (3) The dominant position of different grammatical structures or feature paths was not fully utilized, and the representations obtained from different paths were not dynamically fused, resulting in a decline in model performance. Different grammatical structures or feature paths contribute significantly to sentiment judgment in different scenarios, but existing methods usually assume that the contributions of all paths are equal, or statically select a fixed path as dominant, which limits the model's ability to capture complex sentiment expressions. Summary of the Invention

[0008] To address the technical problems existing in the background art, the present invention adopts the following technical solution: providing a multi-syntax, multi-frequency aspect-level sentiment analysis method, comprising the following analysis steps:

[0009] Step 1: Use a pre-trained general domain embedding method to obtain general word embedding representations, which will serve as the final embedding representation of the sentence;

[0010] Step 2: Use the Mamba model to obtain the long-distance dependencies between each word as a further representation of the sentence;

[0011] Step 3: Use a fully linked graph convolutional neural network to capture semantic information and dependencies in the text, including:

[0012] Step 3.1: Construct a fully connected semantic graph and perform random edge pruning;

[0013] Step 3.2: Use a graph convolutional neural network to further extract semantic information and dependencies;

[0014] Step 4: Construct a semantic graph multi-frequency propagation network to realize semantic roles and abstract semantic encoding, including:

[0015] Step 4.1: Construct a semantic role graph encoding by combining semantic role labeling, abstract semantic representation, and multi-frequency propagation;

[0016] Step 4.2: Design low-pass and high-pass filters to filter signals from node features;

[0017] Step 4.3: Adaptively aggregate messages with different frequencies using filters, and then gradually distribute the multi-frequency information on the graph. Each node receives multi-frequency signals from its K-hop neighbors, and finally obtains the multi-frequency representation, thus realizing the semantic graph encoding of the sentence.

[0018] Step 5: Perform alignment operations to transform the full-chain graph encoding and semantic encoding into a unified feature space, including:

[0019] Step 5.1: Use a multi-head attention mechanism to enhance the feature representation of the full-chain graph encoding representation and the semantic graph encoding representation, and then transform them into a unified feature space;

[0020] Step 5.2: Use the mean squared error loss function to measure the degree of alignment between the two in the unified feature space;

[0021] Step 6: Construct the final sentence representation using a dynamic fusion mechanism, including:

[0022] Step 6.1: Construct dynamic fusion weights using full-chain graph encoding and semantic graph encoding;

[0023] Step 6.2: Use dynamic fusion weights to fuse the full-chain graph encoding representation and the semantic graph encoding representation to construct the final sentence representation;

[0024] Step 7: Generate the sentiment probability distribution using the softmax function;

[0025] Step 8: Construct the corresponding loss function to train the model.

[0026] The specific method for step 1 is as follows:

[0027] Word embedding representation using BERT embedding method Among the words The general domain vector is represented as ,Will As a word vector representation Then the sentence comment is represented as .

[0028] The specific method for step 2 is as follows:

[0029] First, comment on the sentence. Projection and separation are performed, followed by one-dimensional convolution. The calculation formula for this process is as follows:

[0030] ;

[0031] ;

[0032] in, Indicates a gating signal. Indicates the input projection matrix. Indicates the SiLU activation function;

[0033] Then The input is processed by the SSM module, and the calculation formula used in the processing is as follows:

[0034] ;

[0035] ;

[0036] ;

[0037] ;

[0038] ;

[0039] ;

[0040] ;

[0041] ;

[0042] ;

[0043] ;

[0044] in, For time step parameters, Indicates the input projection matrix. Represents element-wise multiplication. Indicates the output projection matrix. , , , These are the parameter matrices for each layer. Indicates the sequence length. These are learnable residual connectivity coefficients;

[0045] Input the results obtained from the SSM module into the gating mechanism:

[0046] ;

[0047] The final output result is as follows:

[0048] ;

[0049] in, To output the projection matrix;

[0050] The output sentence representation is defined as follows: .

[0051] The specific method for step 3.1 is as follows:

[0052] Based on the sentence representation output in step 2, construct an adjacency matrix of a fully connected graph. ,in This represents the edge weight between node i and node j. It is determined by calculating the similarity between word vectors, using the following formula:

[0053] ;

[0054] Perform random edge pruning on the fully connected graph and randomly generate a mask matrix M with the same dimensions as the adjacency matrix A, where each element... The adjacency matrix after pruning is set to 0 if probability p is 0 otherwise. for:

[0055] ;

[0056] in, This represents element-wise multiplication;

[0057] Using a normalized weighted summation method based on the adjacency matrix, the trimmed adjacency matrix is... Convert to node representation For each node i, its new element representation for:

[0058] ;

[0059] in, It is the trimmed adjacency matrix The element in the i-th row and j-th column represents the edge weight between node i and node j. A matrix is ​​formed by combining the representations of all nodes. The expression is:

[0060] .

[0061] The specific method for step 3.2 is as follows:

[0062] Node representations are obtained using graph convolutional neural networks. The expression is:

[0063] ;

[0064] in, These are the learning weights and biases, respectively.

[0065] The expression for the sentence obtained in this way is:

[0066] .

[0067] The specific method for step 4 is as follows:

[0068] Step 4.1: Combine semantic role labeling and abstract semantics to construct a semantic graph, including:

[0069] The sentence comments are subjected to semantic role labeling and abstract semantic representation analysis, and then a semantic graph is constructed. ;

[0070] in, It is a set of words, denoted as {f i}, including entity nodes and abstract semantic nodes, are all words;

[0071] It is a set of edges, consisting of SRL relational edges and AMR logical edges, representing the relationships between predicates and entities and the abstract semantic relationships between words;

[0072] The normalized graph Laplacian matrix can be represented as ;

[0073] Among them, the adjacency matrix , I is the angle matrix, and I is the identity matrix;

[0074] Step 4.2: Implement abstract semantic representation encoding using a multi-frequency filter, including:

[0075] Design a low-pass filter. and a high-pass filter The high-pass filter is equivalent to the normalized graph Laplacian matrix, expressed as:

[0076] ;

[0077] ;

[0078] Step 4.3: Construct a graph learning method to obtain sentence representations, including:

[0079] Using a weighted sum to combine low-frequency and high-frequency messages, the expression is:

[0080] ;

[0081] in, It is the input of layer k. It is a weight matrix for low-frequency and high-frequency information;

[0082] ;

[0083] in, It is a neighboring node of node i. and Let $\mathbf{j}$ be the weight contributions of the low-frequency and high-frequency signals of node $j$ to node $i$, and let $\mathbf{j}$ satisfy the constraints. ;

[0084] ;

[0085] in, It is a cascade operation. It is a trainable weight matrix. It is the hyperbolic tangent function, used to scale values ​​in the range [-1, 1].

[0086] Based on The calculation models the importance of changes in different frequency components. In this case, high-frequency messages dominate, and node i receives the differences between node i and its neighbor j.

[0087] Finally, the multi-frequency information is gradually distributed on the graph by stacking K layers. Each node receives multi-frequency signals from its K-hop neighbors, and the output of the last layer is used as the multi-frequency representation. The abstract semantic graph encoding of the sentence is as follows:

[0088] .

[0089] The specific method for step 5 is as follows:

[0090] The full-link graph is encoded using alignment operations. and semantic graph encoding representation Transform into a unified feature space to obtain a new full-chain graph encoding representation. semantic graph encoding representation ,include:

[0091] Step 5.1: Use a multi-head attention mechanism to enhance the full-chain graph encoding representation. and semantic graph encoding representation The feature representations are then transformed into a unified feature space, expressed as:

[0092] ;

[0093] ;

[0094] in, It is a weight matrix used to transform the output of multi-head attention to a unified feature space;

[0095] Step 5.2: To measure the fully linked graph encoding representation and semantic graph encoding representation The alignment degree in the unified feature space is handled using the mean squared error loss function, expressed as:

[0096] ;

[0097] in, This represents the dimension of the feature representation.

[0098] The specific method for step 6 is as follows:

[0099] The final sentence representation is constructed using a dynamic fusion mechanism. ,include:

[0100] Step 6.1: Represent using aligned full-link graph encoding and semantic graph encoding representation To predict the sentiment score of a decoder composed of MLPs , The calculation formula is:

[0101] ;

[0102] The difference between sentiment scores based on single-path features and fused features is used to indicate the amount of effective information provided by the corresponding path. To further guide attention to weighting;

[0103] Step 6.2: For the weights of single-path features, single-path features The sentiment score of y is inversely proportional to the sentiment score of y. Therefore, we choose the inverse proportional function exp(-kx) and normalization operation. During training, we use the ground truth value of y to calculate the sentiment score of the single-path feature. Converted to sentiment ratio , The calculation formula is:

[0104] ;

[0105] ;

[0106] Where k represents the slope of the function, which can scale the sentiment ratio;

[0107] To unify the representation of feature knowledge enhancement for single paths The length and dimension axes, where, It is the sequence length, d m The vector dimension is represented, and a projector is used to obtain the features of each path. Updated knowledge enhancement representation:

[0108] First, the obtained representation Summation as the first dynamic attention block The input expression is:

[0109] ;

[0110] ;

[0111] in, It consists of two linear layers. and This indicates the length and dimensions of the fusion phase;

[0112] Then, dynamic attention blocks are stacked to form a pipeline, while using the previous block. Output, knowledge augmentation representation And emotional ratio , Use it as input to the next block and obtain its output. The expression is:

[0113] ;

[0114] DAM stands for Dynamic Attention Module.

[0115] In the dynamic attention module, a cross-modal attention module is used to capture single-path feature representations. and fusion feature representation The amount of similar information between them gradually determines the dominant path;

[0116] Using the fused features as Q, and the single-path features as K and V, and applying the layer norm LN, the expression is:

[0117] ;

[0118] Emotional ratio Multiply by the middle representation To further guide dynamic fusion, and to combine the obtained representations and inputs The fusion features are represented by addition to fine-tune the contributions of different syntaxes, expressed as:

[0119] ;

[0120] ;

[0121] Finally, Inputting a multi-head attention and feedforward neural network yields the output of a dynamic attention block. ;

[0122] Use the output of the last block as the final fused representation. .

[0123] The specific method for step 7 is as follows:

[0124] The obtained fusion representation The data is fed into a linear layer and then processed by a softmax function to generate an emotion probability distribution p, expressed as:

[0125] ;

[0126] in, and These are the weights and the biases, respectively.

[0127] The specific method for step 8 is as follows:

[0128] Step 8.1: Define the standard cross-entropy loss used for calculation, expressed as:

[0129] ;

[0130] Where D is the set of all sentiment-aspect pairs, C is the set of sentiment polarities, and θ is the trainable parameters of the model;

[0131] Step 8.2: Define the expression for the total loss as follows:

[0132] ;

[0133] in, and Used to balance the two loss functions and .

[0134] The beneficial effects of this invention compared to existing technologies are as follows: The aspect-level sentiment analysis method based on multi-syntax and multi-frequency proposed in this invention, based on social media comments, uses a full-chain graph convolutional neural network and a semantic graph multi-frequency propagation network to obtain sentence representations from different perspectives, and then uses alignment operations and dynamic fusion mechanisms to obtain a comprehensive sentence representation, thereby achieving aspect-level sentiment analysis, wherein:

[0135] This invention designs a semantic graph multi-frequency propagation network. The nodes in the graph include entity nodes and abstract semantic nodes, all of which are words. The edges are SRL relation edges and AMR logical edges, representing the relationships between predicates and entities, and abstract semantic relationships. Then, low-pass and high-pass filters are designed to filter the signals from the node features. The filters adaptively aggregate messages with different frequencies to obtain a multi-frequency representation, further leading to a comprehensive semantic graph encoding representation. This network can capture deep semantics beyond syntactic structure and can extract key semantic features more stably.

[0136] The alignment operation designed in this invention enhances the feature representation of each branch through an attention mechanism, then transforms them into a unified feature space, and subsequently uses the mean squared error loss function to measure the degree of alignment between them in the unified feature space. This module can supplement and improve the feature representation learned by each branch, further improving classification performance.

[0137] The dynamic fusion mechanism designed in this invention constructs dynamic fusion weights from the feature representations of two paths, and then obtains a fusion representation by dynamic cross-fusion based on the fusion weights, thereby obtaining a comprehensive feature expression and improving the performance of aspect-level sentiment analysis. Attached Figure Description

[0138] The present invention will be further described below with reference to the accompanying drawings:

[0139] Figure 1 This is a flowchart illustrating the framework of the aspect-level sentiment analysis method based on multiple syntaxes and multiple frequencies of this invention.

[0140] Figure 2 This is a flowchart illustrating the framework of the dynamic attention module in this embodiment of the invention. Detailed Implementation

[0141] like Figure 1 and Figure 2 As shown, this invention addresses the problems of current aspect-level sentiment analysis methods by proposing a multi-syntax, multi-frequency aspect-level sentiment analysis scheme. First, a fully-linked graph convolutional neural network is used to capture the syntactic information and dependencies of comments. Then, semantic role labeling and abstract semantic representation are combined to analyze the comments. Multi-frequency propagation is used to capture the semantic relevance of each word in the text, constructing a dual-channel graph neural network. Next, alignment operations are used to transform the fully-linked graph encoding and semantic graph encoding into a unified feature space. Finally, a dynamic fusion mechanism is used to obtain a comprehensive sentence representation, thereby improving the performance of aspect-level sentiment analysis.

[0142] The method of the present invention will be further described in detail below with reference to the accompanying drawings.

[0143] As mentioned earlier, current syntactic-based aspect-level sentiment analysis methods only focus on single syntactic structural information, neglecting deep semantic relationships between words and the guiding role of word frequency information. Therefore, this invention first uses a general domain embedding representation to obtain the initial representation information of the comment, then uses a Mamba model to capture the long-distance dependency information of words in the comment sentence, subsequently uses a full-chain graph convolutional neural network to capture the syntactic information of the comment, and employs a multi-frequency propagation graph learning mechanism to capture the deep latent semantic relationships between comment words, forming a dual-channel neural network. Then, an alignment operation is used to transform the full-chain graph encoding and semantic encoding into a unified feature space, obtaining new text full-chain graph encoding and semantic encoding information, and a dynamic fusion mechanism is used to obtain a more comprehensive vector representation. The aspect-level sentiment analysis method provided by this invention is an incremental framework, including embedding representation, Mamba model, full-chain graph convolutional neural network, semantic graph multi-frequency propagation network, alignment module, dynamic fusion mechanism, and softmax layer, as follows. Figure 1 The diagram shown illustrates its corresponding framework.

[0144] This invention proposes an aspect-level sentiment analysis method based on multiple syntaxes and multiple frequencies. It is primarily implemented using a dual-channel graph neural network, namely a fully-linked graph convolutional neural network and a semantic graph multi-frequency propagation network. The specific operation process is as follows:

[0145] Step 1: Implement the embedding representation of word vectors in the comment sentence using the embedding representation method:

[0146] First, the BERT embedding method is used to implement word embedding representation. This allows different words to typically have different meanings, among which words The general domain vector is represented as ,Will As a word vector representation That is, sentence commentary .

[0147] Step 2: Use the Mamba model to obtain further word representations:

[0148] First, comment on the sentence. Projection and separation are performed, followed by one-dimensional convolution. The calculation formula for this process is as follows:

[0149] ;

[0150] ;

[0151] in, Indicates a gating signal. Indicates the input projection matrix. Indicates the SiLU activation function;

[0152] Then The input is processed by the SSM module, and the calculation formula used in the processing is as follows:

[0153] ;

[0154] ;

[0155] ;

[0156] ;

[0157] ;

[0158] ;

[0159] ;

[0160] ;

[0161] ;

[0162] ;

[0163] in, For time step parameters, Indicates the input projection matrix. Represents element-wise multiplication. Indicates the output projection matrix. , , , These are the parameter matrices for each layer. Indicates the sequence length. These are learnable residual connectivity coefficients.

[0164] Input the results obtained from the SSM module into the gating mechanism:

[0165] ;

[0166] The final output result is as follows:

[0167] ;

[0168] in, This is for outputting the projection matrix.

[0169] Therefore, sentence representation is defined as .

[0170] Step 3: Employ a fully connected graph convolutional neural network to capture semantic information and dependencies in the text. This includes two parts: constructing a fully connected semantic graph and performing edge pruning, and then using the graph convolutional neural network.

[0171] Step 3.1: Construct a fully connected semantic graph and perform random edge pruning:

[0172] Based on the output of the previous step, construct an adjacency matrix for a fully connected graph. ,in This represents the edge weight between node i and node j. It is determined by calculating the similarity between word vectors, using the following formula:

[0173] .

[0174] To prevent overfitting and improve the model's generalization ability, random edge pruning is required on the fully connected graph. A mask matrix M with the same dimensions as the adjacency matrix A is randomly generated, where each element... The adjacency matrix after pruning is set to 0 if probability p is 0 otherwise. for:

[0175] ;

[0176] in, This indicates element-wise multiplication.

[0177] Using a normalized weighted summation method based on the adjacency matrix, the trimmed adjacency matrix is... Convert to node representation For each node i, its new element representation for:

[0178] ;

[0179] in, It is the trimmed adjacency matrix The element in the i-th row and j-th column represents the edge weight between node i and node j. A matrix is ​​formed by combining the representations of all nodes. The expression is:

[0180] .

[0181] Step 3.2: Obtain node representations using a graph convolutional network. The expression is:

[0182] ;

[0183] in, These are the learning weights and biases, respectively.

[0184] The expression for the sentence obtained in this way is:

[0185] .

[0186] Step 4: Combining semantic role labeling and abstract semantic representation, a semantic graph encoding is constructed using a semantic graph multi-frequency propagation network to obtain the semantic relationships of concept words in the sentence. The semantic graph multi-frequency propagation network used includes: semantic graph construction, multi-frequency filtering, and graph learning, among which:

[0187] Step 4.1: Combine semantic role labeling and abstract semantics to construct a semantic graph:

[0188] The sentence comments are subjected to semantic role labeling and abstract semantic representation analysis, and then a semantic graph is constructed. ,in It is a set of nodes (a set of words), denoted as {f i}, including entity nodes and abstract semantic nodes, are all words. It is a set of edges, consisting of SRL relational edges and AMR logical edges, representing the relationships between predicates and entities, and the abstract semantic relationships between words. Adjacency matrix .

[0189] The normalized graph Laplacian matrix can be represented as ;

[0190] in, I is the angle matrix, and I is the identity matrix.

[0191] Step 4.2: Implement abstract semantic representation encoding using a multi-frequency filter:

[0192] Design a low-pass filter. and a high-pass filter :

[0193] ;

[0194] .

[0195] Step 4.3: Construct a graph learning method to obtain sentence representations:

[0196] Using a weighted sum to combine low-frequency and high-frequency messages, the expression is:

[0197] ;

[0198] in, It is the input of layer k. It is a weight matrix for low-frequency and high-frequency information;

[0199] ;

[0200] in, It is a neighboring node of node i. and Let $\mathbf{j}$ be the weight contributions of the low-frequency and high-frequency signals of node $j$ to node $i$, and let $\mathbf{j}$ satisfy the constraints. ;

[0201] ;

[0202] in, It is a cascade operation. It is a trainable weight matrix. It is the hyperbolic tangent function, used to scale values ​​in the range [-1, 1].

[0203] Based on the above calculations The importance of variations in different frequency components can be easily modeled, if In this case, high-frequency messages dominate, and node i receives the differences between node i and its neighbor j.

[0204] Finally, the multi-frequency information is gradually distributed across the graph; by stacking K layers, each node receives multi-frequency signals from its K-hop neighbors, using the output of the last layer as the multi-frequency representation. The abstract semantic graph encoding of the sentence is as follows: .

[0205] Step 5: Encode the full-link graph using alignment operations. and semantic graph encoding representation Transform into a unified feature space to obtain a new full-chain graph encoding representation. semantic graph encoding representation The specific method is as follows:

[0206] Step 5.1: Use a multi-head attention mechanism to enhance the full-chain graph encoding representation. and semantic graph encoding representation The feature representations are then transformed into a unified feature space, expressed as:

[0207] ;

[0208] ;

[0209] in, It is a weight matrix used to transform the output of multi-head attention to a unified feature space.

[0210] Step 5.2: To measure the fully linked graph encoding representation and semantic graph encoding representation The alignment in the unified feature space is handled using the mean squared error (MSE) loss function, expressed as:

[0211] ;

[0212] in, This represents the dimension of the feature representation.

[0213] Step 6: Construct the final sentence representation using a dynamic fusion mechanism. The specific method is as follows:

[0214] Step 6.1: Represent using aligned full-link graph encoding and semantic graph encoding representation To predict the sentiment score of a decoder composed of MLPs , The calculation formula is:

[0215] ;

[0216] Since the difference between sentiment scores of single-path features and fused features can indicate the amount of effective information provided by the corresponding path, sentiment scores are used. To further guide attention to weighting.

[0217] Step 6.2: For the weights of single-path features, single-path features The sentiment score of y is inversely proportional to the sentiment score of y. Therefore, we choose the inverse proportional function exp(-kx) and normalization operation. During training, we use the ground truth value of y to calculate the sentiment score of the single-path feature. Converted to sentiment ratio , The calculation formula is:

[0218] ;

[0219] ;

[0220] Here, k represents the slope of the function, which can scale the sentiment ratio.

[0221] Furthermore, in order to unify the representation used for single-path feature knowledge enhancement The length and dimension axes, where, It is the sequence length, d m The vector dimension is represented, and a projector is used to obtain the features of each path. Updated knowledge-enhanced representation. First, the obtained representation... Summation as the first dynamic attention block The input expression is:

[0222] ;

[0223] ;

[0224] in, It consists of two linear layers. and This indicates the length and dimensions of the fusion phase;

[0225] Then, dynamic attention blocks are stacked to form a pipeline, while using the previous block. Output, knowledge augmentation representation And emotional ratio , Use it as input to the next block and obtain its output. The expression is:

[0226] ;

[0227] DAM stands for Dynamic Attention Module.

[0228] In the dynamic attention module, a cross-modal attention module (CAttn) is first introduced, which captures single-path feature representations. and fusion feature representation The dominant path is gradually determined by the similarity of information between them. Since the Q of the attention is used to specify the location of the attention, the fused features are used as Q, the single-path features are used as K and V, and the layer norm (LN) is performed, expressed as:

[0229] ;

[0230] Next, due to the emotional ratio This can further guide dynamic fusion, so it is multiplied by the intermediate representation. .

[0231] Then, the obtained representation and input The fusion features are represented by addition to fine-tune the contributions of different syntaxes, expressed as:

[0232] ;

[0233] ;

[0234] Finally, Inputting a multi-head attention and feedforward neural network yields the output of a dynamic attention block. .

[0235] Use the output of the last block as the final fused representation. .

[0236] Step 7: Generate the sentiment probability distribution using the softmax function: The obtained fused representation The data is fed into a linear layer and then processed by a softmax function to generate an emotion probability distribution p, expressed as:

[0237] ;

[0238] in, and These are the weights and the biases, respectively.

[0239] Step 8: Construct the corresponding loss function to train the model.

[0240] Step 8.1: Define the standard cross-entropy loss used for calculation, expressed as:

[0241] ;

[0242] Where D is the set of all sentiment-aspect pairs, C is the set of sentiment polarities, and θ is the trainable parameters of the model.

[0243] Step 8.2: Define the expression for the total loss as follows:

[0244] ;

[0245] in, and Used to balance the two loss functions and .

[0246] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-syntax, multi-frequency aspect-level sentiment analysis method, characterized in that: The analysis includes the following steps: Step 1: Use a pre-trained general domain embedding method to obtain general word embedding representations, which will serve as the final embedding representation of the sentence; Step 2: Use the Mamba model to obtain the long-distance dependencies between each word as a further representation of the sentence; Step 3: Use a fully linked graph convolutional neural network to capture semantic information and dependencies in the text, including: Step 3.1: Construct a fully connected semantic graph and perform random edge pruning; Step 3.2: Use a graph convolutional neural network to further extract semantic information and dependencies; Step 4: Construct a semantic graph multi-frequency propagation network to realize semantic roles and abstract semantic encoding, including: Step 4.1: Construct a semantic role graph encoding by combining semantic role labeling, abstract semantic representation, and multi-frequency propagation, including: The sentence comments are subjected to semantic role labeling and abstract semantic representation analysis, and then a semantic graph is constructed. ; in, It is a set of words, denoted as {f i }, including entity nodes and abstract semantic nodes, are all words; It is a set of edges, consisting of SRL relational edges and AMR logical edges, representing the relationships between predicates and entities and the abstract semantic relationships between words; The normalized graph Laplace matrix is ​​represented as ; Among them, the adjacency matrix , I is the angle matrix, and I is the identity matrix; Step 4.2: Design low-pass and high-pass filters to filter signals from node features, including: Design a low-pass filter. and a high-pass filter The high-pass filter is equivalent to the normalized graph Laplacian matrix, expressed as: ； ； Step 4.3: Adaptively aggregate messages with different frequencies using filters, then gradually distribute the multi-frequency information on the graph. Each node receives multi-frequency signals from its K-hop neighbors, ultimately obtaining a multi-frequency representation and realizing semantic graph encoding of the sentence, including: Using a weighted sum to combine low-frequency and high-frequency messages, the expression is: ； in, It is the input of layer k. It is a weight matrix for low-frequency and high-frequency information; ； in, It is a neighboring node of node i. and The weighted contributions of the low-frequency and high-frequency signals of node j to node i satisfy the constraints. ; ； in, It is a cascade operation. It is a trainable weight matrix. It is the hyperbolic tangent function, used to scale values ​​in the range [-1, 1]. Based on The calculation models the importance of changes in different frequency components. In this case, high-frequency messages dominate, and node i receives the differences between node i and its neighbor j. Finally, the multi-frequency information is gradually distributed on the graph by stacking K layers. Each node receives multi-frequency signals from its K-hop neighbors, and the output of the last layer is used as the multi-frequency representation. The abstract semantic graph encoding of the sentence is as follows: ； Step 5: Perform alignment operations to transform the full-chain graph encoding and semantic encoding into a unified feature space, including: Step 5.1: Use a multi-head attention mechanism to enhance the feature representation of the full-chain graph encoding representation and the semantic graph encoding representation, and then transform them into a unified feature space; Step 5.2: Use the mean squared error loss function to measure the degree of alignment between the two in the unified feature space; Step 6: Construct the final sentence representation using a dynamic fusion mechanism, including: Step 6.1: Construct dynamic fusion weights using full-chain graph encoding and semantic graph encoding; Step 6.2: Use dynamic fusion weights to fuse the full-chain graph encoding representation and the semantic graph encoding representation to construct the final sentence representation; Step 7: Generate the sentiment probability distribution using the softmax function; Step 8: Construct the corresponding loss function to train the model.

2. The aspect-level sentiment analysis method based on multiple syntaxes and multiple frequencies according to claim 1, characterized in that: The specific method for step 1 is as follows: Word embedding representation using BERT embedding method Among the words The general domain vector is represented as ,Will As a word vector representation Then the sentence comment is represented as .

3. The aspect-level sentiment analysis method based on multiple syntaxes and multiple frequencies according to claim 1, characterized in that: The specific method for step 2 is as follows: First, comment on the sentence. Projection and separation are performed, followed by one-dimensional convolution. The calculation formula for this process is as follows: ； ； in, Indicates a gating signal. Indicates the input projection matrix. Indicates the SiLU activation function; Then The input is processed by the SSM module, and the calculation formula used in the processing is as follows: ； ； ； ； ； ； ； ； ； ； in, For time step parameters, Indicates the input projection matrix. Represents element-wise multiplication. Indicates the output projection matrix. , , , These are the parameter matrices for each layer. Indicates the sequence length. These are learnable residual connectivity coefficients; Input the results obtained from the SSM module into the gating mechanism: ； The final output result is as follows: ； in, To output the projection matrix; The output sentence representation is defined as follows: .

4. The aspect-level sentiment analysis method based on multiple syntaxes and multiple frequencies according to claim 1, characterized in that: The specific method for step 3.1 is as follows: Based on the sentence representation output in step 2, construct an adjacency matrix of a fully connected graph. ,in This represents the edge weight between node i and node j. It is determined by calculating the similarity between word vectors, using the following formula: ； Perform random edge pruning on the fully connected graph and randomly generate a mask matrix M with the same dimensions as the adjacency matrix A, where each element... The adjacency matrix after pruning is set to 0 if probability p is 0 otherwise. for: ； in, This represents element-wise multiplication; Using a normalized weighted summation method based on the adjacency matrix, the trimmed adjacency matrix is... Convert to node representation For each node i, its new element representation for: ； in, It is the trimmed adjacency matrix The element in the i-th row and j-th column represents the edge weight between node i and node j. A matrix is ​​formed by combining the representations of all nodes. The expression is: 。 5. The aspect-level sentiment analysis method based on multiple syntaxes and multiple frequencies according to claim 1, characterized in that: The specific method for step 3.2 is as follows: Node representations are obtained using graph convolutional neural networks. The expression is: ； in, These are the learning weights and biases, respectively. The expression for the sentence obtained in this way is: 。 6. The aspect-level sentiment analysis method based on multiple syntaxes and multiple frequencies according to claim 1, characterized in that: The specific method for step 5 is as follows: The full-link graph is encoded using alignment operations. and semantic graph encoding representation Transform into a unified feature space to obtain a new full-chain graph encoding representation. semantic graph encoding representation ,include: Step 5.1: Use a multi-head attention mechanism to enhance the full-chain graph encoding representation. and semantic graph encoding representation The feature representations are then transformed into a unified feature space, expressed as: ； ； in, It is a weight matrix used to transform the output of multi-head attention to a unified feature space; Step 5.2: To measure the fully linked graph encoding representation and semantic graph encoding representation The alignment degree in the unified feature space is handled using the mean squared error loss function, expressed as: ； in, This represents the dimension of the feature representation.

7. The aspect-level sentiment analysis method based on multi-syntax and multi-frequency as described in claim 1, characterized in that: The specific method for step 6 is as follows: The final sentence representation is constructed using a dynamic fusion mechanism. ,include: Step 6.1: Represent using aligned full-link graph encoding and semantic graph encoding representation To predict the sentiment score of a decoder composed of MLPs , The calculation formula is: ； The difference between sentiment scores based on single-path features and fused features is used to indicate the amount of effective information provided by the corresponding path. To further guide attention to weighting; Step 6.2: For the weights of single-path features, single-path features The sentiment score of y is inversely proportional to the sentiment score of y. Therefore, we choose the inverse proportional function exp(-kx) and normalization operation. During training, we use the ground truth value of y to calculate the sentiment score of the single-path feature. Converted to sentiment ratio , The calculation formula is: ； ； Where k represents the slope of the function, used to scale the sentiment ratio; To unify the representation of feature knowledge enhancement for single paths The length and dimension axes, where, It is the sequence length, d m The vector dimension is represented, and a projector is used to obtain the features of each path. Updated knowledge enhancement representation: First, the obtained representation Summation as the first dynamic attention block The input expression is: ； ； in, It consists of two linear layers. and This indicates the length and dimensions of the fusion phase; Then, dynamic attention blocks are stacked to form a pipeline, while using the previous block. Output, knowledge augmentation representation And emotional ratio , Use it as input to the next block and obtain its output. The expression is: ； DAM stands for Dynamic Attention Module. In the dynamic attention module, a cross-modal attention module is used to capture single-path feature representations. and fusion feature representation The amount of similar information between them gradually determines the dominant path; Using the fused features as Q, and the single-path features as K and V, and applying the layer norm LN, the expression is: ； Emotional ratio Multiply by the middle representation To further guide dynamic fusion, and to combine the obtained representations and inputs The fusion features are represented by addition to fine-tune the contributions of different syntaxes, expressed as: ； ； Finally, Inputting a multi-head attention and feedforward neural network yields the output of a dynamic attention block. ; Use the output of the last block as the final fused representation. .

8. The aspect-level sentiment analysis method based on multi-syntax and multi-frequency as described in claim 1, characterized in that: The specific method for step 7 is as follows: The obtained fusion representation The data is fed into a linear layer and then processed by a softmax function to generate an emotion probability distribution p, expressed as: ； in, and These are the weights and the biases, respectively.

9. The aspect-level sentiment analysis method based on multi-syntax and multi-frequency as described in claim 6, characterized in that: The specific method for step 8 is as follows: Step 8.1: Define the standard cross-entropy loss used for calculation, expressed as: ； Where D is the set of all sentiment-aspect pairs, C is the set of sentiment polarities, and θ is the trainable parameters of the model; Step 8.2: Define the expression for the total loss as follows: ； in, and Used to balance the two loss functions and .

Images