Fine-grained sentiment analysis method based on denoising template and abstract semantic representation information

Abstract

The invention discloses a fine-grained sentiment analysis method based on a denoising template and abstract semantic representation information, and belongs to the technical field of sentiment analysis. The method specifically comprises the following steps: 1, inputting a to-be-trained statement sequence X into a BERT-based self-defined template compiler, and generating a high-dimensional feature vector Z1 carrying aspect word information through a denoising template; 2, analyzing the AMR map from the statement sequence X by using an analyzer SPRING, and performing alignment through an aligner LEAMR to generate the AMR map; 3, converting the relationship between nodes and edges in the AMR into an adjacent matrix, and importing the adjacent matrix into BERT as an encoder to obtain a high-dimensional feature vector R; 4, constructing a semantic enhanced attention mechanism module by integrating a vector R carrying semantic information through an attention mechanism module, and inputting Z1 into an encoder to obtain a high-dimensional feature vector R ', and 5, finally inputting R' into a decoder for further decoding to generate a final output vector W.

Description

Technical Field

[0001] The present invention belongs to the technical field of sentiment analysis, and in particular relates to a fine-grained sentiment analysis method based on a denoising template and abstract semantic representation information. Background Art

[0002] With the rapid development of information technology, there is a strong demand for hot spot detection, data monitoring and sentiment analysis of public opinion by calculating the opinions, emotions, evaluations and attitudes in texts on the Internet. The research technology for sentiment analysis of the whole text has become relatively mature and has achieved good performance. However, the overall sentiment analysis of the text will cover up its details, and the overall sentiment cannot reflect people's fine-grained emotional expression of the opinion target. Therefore, in order to perform a more complete sentiment analysis, fine-grained aspect-level sentiment analysis technology has emerged. At present, there are two major challenges in fine-grained sentiment analysis: (1) Aspect-level sentiment analysis needs to obtain aspect information of the evaluation object in the text. The evaluation object aspect may appear explicitly in the text or may be implicitly included in the text semantics. Because the form and number of aspects are uncertain, aspect-level sentiment analysis technology must not only analyze the explicit language expression structure, but also have a deep understanding of the implicit semantic expression. (2) Aspect-level sentiment analysis needs to determine the contextual scope of the emotion expressed for each evaluation object. Natural language is not only coherent and flexible in expression, but also very common in expressions such as pronouns. Aspect-level sentiment analysis technology needs to correctly understand the semantic information of the text word level and sentence level in order to correctly define the context scope for expressing emotions about each aspect of the evaluation object. Solving the above two challenges will also solve the problem of low accuracy of aspect-level sentiment analysis tasks.

[0003] In response to these challenges, researchers have proposed many methods and techniques to improve the performance and efficiency of aspect-level sentiment analysis tasks, mainly including research in model optimization, algorithm improvement, computational acceleration, noise suppression, etc. Wu et al. proposed a grid tag scheme (GTS), where for each word pair, the model predicts whether they belong to the same aspect, the same opinion, an aspect-opinion pair, or none of the above. Then, the original word pair extraction task was transformed into a unified TokenClass problem, but the processing efficiency was low and the effect was not optimal. Zhao et al. regarded the problem as a joint extraction of terms and relations, and designed a span-based multi-task learning (SpanMlt) framework to jointly extract aspects / opinions and pairing relations, but did not involve a language model, and the model had a strong conditional independence assumption, which actually reduced the accuracy of sentiment recognition. Chen et al. proposed a model with two channels to extract aspect / opinion terms and relations respectively. Two synchronization mechanisms were further designed to achieve information interaction between the two channels, but the computational cost may be very high. Schmitt et al. added a dimension called "N / A" to indicate whether the category appears in the sentence, so as to handle it in a unified way. Liu et al. adopted the Seq2Seq modeling paradigm to solve the ACSA problem. Based on the pre-trained generative model and using natural language sentences to represent the desired output, its performance is better than the previous classification type model. Experimental results show that this paradigm can better utilize pre-trained knowledge and has great advantages in few-sample and zero-sample situations. However, the model does not capture the connection between word-level and letter-level semantics well, resulting in a certain degree of reduced accuracy. Summary of the invention

[0004] Based on the defects and shortcomings of the existing methods for the above-mentioned aspect-level sentiment classification tasks, the present invention proposes a fine-grained sentiment analysis method based on denoising templates and abstract semantic representation information, which can improve the accuracy of aspect-level sentiment analysis tasks.

[0005] The present invention is implemented by the following technical scheme: a fine-grained sentiment analysis method based on denoising template and abstract semantic representation information, comprising:

[0006] Step 1: Input the sentence sequence X to be trained into the custom template compiler based on BERT, and generate a high-dimensional feature vector Z carrying aspect word information through the denoising template. 1 ;

[0007] The sentence sequence x to be trained i Construct a specific template prompt word to extract the distance information between aspect items. i All aspects included [A i ] are extracted and arranged into corresponding relationships. Add [dis] to add the prompt as a suffix to each x i In the above example, we get the input sequence X 1 .

[0008] The template denoising method is used to optimize the deviation caused by the introduction of the prompt word. i and X i [pos] is added at the end to indicate the position deviation information, and then the original sequence and the templated sequence are directly fed to BERT to calculate the template deviation h i For example, if X i If there are 5 additional tokens, add 5 to the position id of the template token after [X] to ensure that the position id of the template is the same. 1 -h i X is the sentence representation after denoising 2 .

[0009] Furthermore, BERT is used to train X 2 Encode and produce a series of hidden states: Since BERT has a strong ability to model sentence pairs, the [dis] added in step 1 is generated by BERT. dis It not only contains the information of aspects and context, but also the dependencies between aspect items. Therefore, the original aspect item hidden state plus the average distance to other aspect items get As the initial contextualized aspect item state Then use Replace H D Middle aspect words As the hidden state, we get the initial context hidden state sequence

[0010] Step 2: Use the parser SPRING to parse the AMR graph from the sentence sequence X and align it through the aligner LEAMR to generate the AMR graph;

[0011] A series of context hidden states and description hidden states have been obtained in the above step 1. In order to incorporate as many correspondences between favorable aspect items and sentiment items as possible without losing the original semantics obtained from BERT, the present invention replaces the traditional syntactic structure dependency tree with AMR to better extract the semantic information contained in the context. As a semantic representation method, it aims to simplify the processing of semantic tasks by extracting key information (such as entities, relationships, etc.) in sentences. AMR removes information in the text that is irrelevant to semantic tasks by explicitly representing the propositional structure of the sentence, making important information more prominent, thereby theoretically simplifying the difficulty of model learning to perform these tasks. First, the AMR graph is parsed from the input sentence using the off-the-shelf parser SPRING. Secondly, the AMR is aligned by the aligner LEAMR. Based on the alignment, the AMR relationship between words in the sentence is reconstructed to obtain a converted AMR with words as nodes.

[0012] Step 3: Convert the relationship between nodes and edges in AMR into an adjacency matrix and import it into BERT as an encoder to obtain a high-dimensional feature vector R.

[0013] After alignment, we need to obtain the embedding of AMR information so that we can learn it through the model later. For the words in the sentence, we also use BERT as the encoder to get the context embedding H A ={h1, h2, .., h n For the edges in AMR, the relationship between nodes is represented as an adjacency matrix R = {r ij |1≤i,j≤n}, where r ij It is the word w i and word w j The embedding of the edge label between. If in AMR w i and w j If there is no edge between them, then give r ij Assign an embedding represented as "none". Edge label embedding is also obtained from the pre-trained model. In order to integrate information from AMR and sentences to obtain optimized relational features, the present invention implements global features by broadcasting local information along the path in the graph, which requires adding a linear activation layer to aggregate semantically relevant information. The specific details are shown in the following formulas (1)(2)(3)(4)(5):

[0014] r′ ij =LayerNorm(r ij ) (1)

[0015] g ij =sigmoid(Linear(r′ ij )) (2)

[0016] a ij =g ij ⊙Linear(r′ ij ) (3)

[0017]

[0018] where r′ ij Represents r ij The standardized output, g ij Represents the output obtained by the activation function sigmoid, a ij Indicates the output of the inner product of the obtained results. Represents the output obtained through the linear layer and residual connection.

[0019] In practical applications, since the aspect-level sentiment analysis task pays more attention to neighbor information, and the BERT encoder with attention mechanism makes the features comprehensive enough, it can achieve quite good results after aggregation. Finally, the aggregate R F The output is converted into a relational attention weight matrix Used for subsequent calculations.

[0020] Step 4: Construct a semantically enhanced attention mechanism module by integrating the attention mechanism module and the vector R carrying semantic information. 1 Input into the encoder to obtain a high-dimensional feature vector R';

[0021] The obtained embedding code is calculated through the classic self-attention mechanism module formula (6) to calculate the attention weight:

[0022]

[0023] where Q and K are d-dimensional input vectors, and W Q and W K Yes and R d×d Next, in order to construct the self-attention mechanism for relational enhancement, the relational attention weight matrix R in AMR is F Added to the original attention weight, it can be expressed as formula (7):

[0024]

[0025] Join R F , the attention output is further guided by the semantic information from AMR, thereby improving the effective attention to semantic keywords. Finally, the relation-enhanced aspect item representation matrix is ​​obtained For subsequent classification.

[0026] Step 5: Input R' into the decoder for further decoding to generate the final output vector W.

[0027] The model is trained and the probability of mapping to three sentiment polarities is calculated through a fully connected softmax layer as formula (8):

[0028]

[0029] Using cross entropy loss as the objective function is formula (9):

[0030]

[0031] Where y is the true sentiment polarity, D contains all matching pairs of aspect items and sentiment items, and C contains all sentiment polarities.

[0032] The beneficial effects of the present invention are:

[0033] The fine-grained sentiment analysis method proposed in the present invention is applicable to any scenario where it is necessary to extract aspect words in a sentence and their corresponding sentiment polarity. It extracts the distance information of the aspect items between sentences hidden in the pre-trained model through a specific denoising template, and fuses the abstract semantic representation (AMR) to build a self-attention mechanism network with enhanced relationships, forming a method that fuses denoising template information and abstract semantic representation information. This method surpasses the accuracy of previous methods based on single-item grammatical dependency trees and single-item pre-trained models, and has extremely high application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the process of the present invention;

[0035] Figure 2 It is the overall structural framework diagram of the present invention;

[0036] Figure 3 Generate AMR graph algorithm flow chart. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] A fine-grained sentiment analysis method based on denoising templates and abstract semantic representation information, including:

[0039] Step 1: Input the sentence sequence X to be trained into the custom template compiler based on BERT, and generate a high-dimensional feature vector Z carrying aspect word information through the denoising template. 1 ;

[0040] The sentence sequence x to be trained i Construct a specific template prompt word to extract the distance information between aspect items. i All aspects included [A i ] are extracted and arranged into corresponding relationships. Add [dis] to add the prompt as a suffix to each x i In the above example, we get the input sequence X 1 .

[0041] The template denoising method is used to optimize the deviation caused by the introduction of the prompt word. i and X i [pos] is added at the end to indicate the position deviation information, and then the original sequence and the templated sequence are directly fed to BERT to calculate the template deviation h i For example, if X i If there are 5 additional tokens, add 5 to the position id of the template token after [X] to ensure that the position id of the template is the same. 1 -h i X is the sentence representation after denoising 2 .

[0042] Furthermore, BERT is used to train X 2 Encode and produce a series of hidden states: Since BERT has a strong ability to model sentence pairs, the [dis] added in step 1 is generated by BERT. dis It not only contains the information of aspects and context, but also the dependencies between aspect items. Therefore, the original aspect item hidden state plus the average distance to other aspect items get As the initial contextualized aspect item state Then use Replace H D Middle aspect words As the hidden state, we get the initial context hidden state sequence

[0043] Step 2: Use the parser SPRING to parse the AMR graph from the sentence sequence X and align it through the aligner LEAMR to generate the AMR graph;

[0044] A series of context hidden states and description hidden states have been obtained in the above step 1. In order to incorporate as many correspondences between favorable aspect items and sentiment items as possible without losing the original semantics obtained from BERT, the present invention replaces the traditional syntactic structure dependency tree with AMR to better extract the semantic information contained in the context. As a semantic representation method, it aims to simplify the processing of semantic tasks by extracting key information (such as entities, relationships, etc.) in sentences. AMR removes information in the text that is irrelevant to semantic tasks by explicitly representing the propositional structure of the sentence, making important information more prominent, thereby theoretically simplifying the difficulty of model learning to perform these tasks. First, the AMR graph is parsed from the input sentence using the off-the-shelf parser SPRING. Secondly, the AMR is aligned by the aligner LEAMR. Based on the alignment, the AMR relationship between words in the sentence is reconstructed to obtain a converted AMR with words as nodes.

[0045] Step 3: Convert the relationship between nodes and edges in AMR into an adjacency matrix and import it into BERT as an encoder to obtain a high-dimensional feature vector R.

[0046] After alignment, we need to obtain the embedding of AMR information so that we can learn it through the model later. For the words in the sentence, we also use BERT as the encoder to get the context embedding H A ={h1, h2, .., h n For the edges in AMR, the relationship between nodes is represented as an adjacency matrix R = {r ij |1≤i,j≤n}, where r ij It is the word w i and word w j The embedding of the edge label between. If in AMR w i and w j If there is no edge between them, then give r ij Assign an embedding denoted as "none". Edge label embeddings are also obtained from the pre-trained model.

[0047] In order to integrate the information from AMR and sentences to obtain optimized relational features, the present invention realizes global features by broadcasting local information along the path in the graph, which requires adding a linear activation layer to aggregate semantic related information. Specifically, it is shown in the following formulas (1)(2)(3)(4)(5):

[0048] r′ ij =LayerNorm(r ij ) (1)

[0049] g ij=sigmoid(Linear(r′ ij )) (2)

[0050] a ij =g ij ⊙Linear(r′ ij ) (3)

[0051]

[0052] where r′ ij Represents r ij The standardized output, g ij Represents the output obtained by the activation function sigmoid, a ij Indicates the output of the inner product of the obtained results. Represents the output obtained through the linear layer and residual connection.

[0053] In practical applications, since the aspect-level sentiment classification task pays more attention to neighbor information, and the BERT encoder with attention mechanism makes the features comprehensive enough, it can achieve quite good results after aggregation. Finally, the aggregated R F The output is converted into a relational attention weight matrix Used for subsequent calculations.

[0054] Step 4: Construct a semantically enhanced attention mechanism module by integrating the attention mechanism module and the vector R carrying semantic information. 1 Input into the encoder to obtain a high-dimensional feature vector R';

[0055] The obtained embedding code is calculated through the classic self-attention mechanism module formula (6) to calculate the attention weight:

[0056]

[0057] where Q and K are d-dimensional input vectors, and W Q and W K Yes and R d×d Next, in order to construct the self-attention mechanism for relational enhancement, the relational attention weight matrix R in AMR is F Added to the original attention weight, it can be expressed as formula (7):

[0058]

[0059] Join R F , the attention output is further guided by the semantic information from AMR, thereby improving the effective attention to semantic keywords. Finally, the relation-enhanced aspect item representation matrix is ​​obtained For subsequent classification.

[0060] The specific algorithm for generating the AMR graph algorithm is as follows Figure 3 shown.

[0061] Step 5: Input R' into the decoder for further decoding to generate the final output vector W.

[0062] The model is trained and the probability of mapping to three sentiment polarities is calculated through a fully connected softmax layer as formula (8):

[0063]

[0064] Using cross entropy loss as the objective function is formula (9):

[0065]

[0066] Where y is the true sentiment polarity, D contains all matching pairs of aspect items and sentiment items, and C contains all sentiment polarities.

[0067] Embodiment 1:

[0068] This application embodiment proposes a fine-grained sentiment analysis method, see Figure 1 As shown, the method includes:

[0069] S101: Input the sentence sequence X to be trained into the custom template compiler based on BERT, and generate a high-dimensional feature vector Z carrying aspect word information through the denoising template. 1 ;

[0070] The sentence sequence x to be trained i Construct a specific template prompt word to extract the distance information between aspect items. i All aspects included [A i ] are extracted and arranged into corresponding relationships. Add [dis] to add the prompt as a suffix to each x i In the above example, we get the input sequence X 1 .

[0071] The template denoising method is used to optimize the deviation caused by the introduction of the prompt word. i and X i [pos] is added at the end to indicate the position deviation information, and then the original sequence and the templated sequence are directly fed to BERT to calculate the template deviation h i For example, if X iIf there are 5 additional tokens, add 5 to the position id of the template token after [X] to ensure that the position id of the template is the same. 1 -h i X is the sentence representation after denoising 2 .

[0072] Furthermore, BERT is used to train X 2 Encode and produce a series of hidden states: Since BERT has a strong ability to model sentence pairs, the [dis] added in step 1 is generated by BERT. dis It not only contains the information of aspects and context, but also the dependencies between aspect items. Therefore, the original aspect item hidden state plus the average distance to other aspect items get As the initial contextualized aspect item state Then use Replace H D Middle aspect words As the hidden state, we get the initial context hidden state sequence

[0073] S102, using the parser SPRING to parse the AMR graph from the sentence sequence X and aligning it through the aligner LEAMR to generate an AMR graph;

[0074] A series of context hidden states and description hidden states have been obtained in the above S101. In order to incorporate as many correspondences between favorable aspect items and sentiment items as possible without losing the original semantics obtained from BERT, the present invention replaces the traditional syntactic structure dependency tree with AMR to better extract the semantic information contained in the context. As a semantic representation method, it aims to simplify the processing of semantic tasks by extracting key information (such as entities, relationships, etc.) in sentences. AMR removes information in the text that is irrelevant to semantic tasks by explicitly representing the propositional structure of the sentence, making important information more prominent, thereby theoretically simplifying the difficulty of model learning to perform these tasks. First, the AMR graph is parsed from the input sentence using the existing parser SPRING. Secondly, the AMR is aligned by the aligner LEAMR. Based on the alignment, the AMR relationship between words in the sentence is reconstructed to obtain a conversion AMR with words as nodes.

[0075] S103, converting the relationship between nodes and edges in AMR into an adjacency matrix and importing BERT as an encoder to obtain a high-dimensional feature vector R;

[0076] After alignment, we need to obtain the embedding of AMR information so that we can learn it through the model later. For the words in the sentence, we also use BERT as the encoder to get the context embedding H A ={h1, h2, .., h n For the edges in AMR, the relationship between nodes is represented as an adjacency matrix R = {r ij |1≤i,j≤n}, where r ij It is the word w i and word w j The embedding of the edge label between. If in AMR w i and w j If there is no edge between them, then give r ij Assign an embedding represented as "none". Edge label embedding is also obtained from the pre-trained model. In order to integrate information from AMR and sentences to obtain optimized relational features, the present invention implements global features by broadcasting local information along the path in the graph, which requires adding a linear activation layer to aggregate semantically relevant information. The specific details are shown in the following formulas (1)(2)(3)(4)(5):

[0077] r′ ij =LayerNorm(r ij ) (1)

[0078] g ij =sigmoid(Linear(r′ ij )) (2)

[0079] a ij = gij ⊙Linear(r′ ij ) (3)

[0080]

[0081] where r′ ij Represents r ij The standardized output, g ij Represents the output obtained by the activation function sigmoid, a ij Indicates the output of the inner product of the obtained results. Represents the output obtained through the linear layer and residual connection.

[0082] In practical applications, since the aspect-level sentiment classification task pays more attention to neighbor information, and the BERT encoder with attention mechanism makes the features comprehensive enough, it can achieve quite good results after aggregation. Finally, the aggregated R FThe output is converted into a relational attention weight matrix Used for subsequent calculations.

[0083] S104, construct a semantically enhanced attention mechanism module by integrating the attention mechanism module and the vector R carrying semantic information, and convert Z 1 Input into the encoder to obtain a high-dimensional feature vector R';

[0084] The obtained embedding code is calculated through the classic self-attention mechanism module formula (6) to calculate the attention weight:

[0085]

[0086] where Q and K are d-dimensional input vectors, and W Q and W K Yes and R d×d Next, in order to construct the self-attention mechanism for relational enhancement, the relational attention weight matrix R in AMR is F Added to the original attention weight, it can be expressed as formula (7):

[0087]

[0088] Join R F , the attention output is further guided by the semantic information from AMR, thereby improving the effective attention to semantic keywords. Finally, the relation-enhanced aspect item representation matrix is ​​obtained For subsequent classification.

[0089] S105 , input R′ into the decoder for further decoding to generate the final output vector W.

[0090] The model is trained and the probability of mapping to three emotional polarities is calculated through a fully connected 8-of-10 max layer as formula (8):

[0091]

[0092] Using cross entropy loss as the objective function is formula (9):

[0093]

[0094] Where y is the true sentiment polarity, D contains all matching pairs of aspect items and sentiment items, and C contains all sentiment polarities.

[0095] Specifically, when training the fine-grained sentiment analysis method based on denoising templates and abstract semantic representation information, the dataset used is Lap14, and the accuracy (Acc) and Macro-F1 (F1) are used as evaluation indicators. The BERT version uses BERT-base uncased, and the initial learning rate is 1×10 -5 , batch size is set to 256, dropout rate is set to 0.5, and the relation-enhanced attention mechanism module encoder uses a 12-layer block attention mechanism. The decoder has a similar stacked block structure with 6 layers and only Transformer blocks with full attention. The model is trained using the Adam optimizer with default configuration. For the feedforward layer, a feedforward layer of dimension 2048 is used and softmax is applied as the activation function. In self-attention, the relative position embedding is enhanced to the input. The model is trained for a total of 1500 epochs. The evaluation is performed on the model at the best checkpoint.

[0096] Experiments show that the Acc of the trained fine-grained sentiment analysis method based on denoising templates and abstract semantic representation information is 83.43% and the F1 is 79.76%. The results prove the necessity, effectiveness and accuracy of the fine-grained sentiment analysis method based on denoising templates and abstract semantic representation information.

[0097] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can well understand and use the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A fine-grained sentiment analysis method based on denoising templates and abstract semantic representation information, characterized by: The following steps are involved: Step 1: Input the sentence sequence X to be trained into the custom template compiler based on BERT, and generate a high-dimensional feature vector Z carrying aspect word information through the denoising template. 1 ; Step 2: Use the parser SPRING to parse the AMR graph from the sentence sequence X and align it through the aligner LEAMR to generate the AMR graph; Step 3: Convert the relationship between nodes and edges in AMR into an adjacency matrix and import it into BERT as an encoder to obtain a high-dimensional feature vector R. Step 4: Construct a semantically enhanced attention mechanism module by integrating the attention mechanism module and the vector R carrying semantic information. 1 The high-dimensional feature vector R' is obtained from the input encoder; Step 5: Input R' into the decoder for further decoding to generate the final output vector W.

2. According to claim 1, a fine-grained sentiment analysis method based on denoising template and abstract semantic representation information is characterized in that: In the step 1), the specific method is: 1.1) Treat the sentence sequence x for training i Construct a specific template prompt word to extract the distance information between aspect items. i All aspects included [A i ] are extracted and arranged into corresponding relationships. Add [dis] to add the prompt as a suffix to each x i In the above example, we get the input sequence X 1 ; 1.2) Use template denoising method to optimize the deviation caused by introducing prompt words, in x i and X i [pos] is added at the end to indicate the position deviation information, and then the original sequence and the templated sequence are directly fed to BERT to calculate the template deviation h i ; 1.3) Use BERT to train X 2 Encode and produce a series of hidden states: Since BERT has a strong ability to model sentence pairs, the added [dis] is generated by BERT dis Contains information about aspects and context, as well as dependencies between aspect items. It adds the original hidden state of the aspect item to the average distance to other aspect items. get As the initial contextualized aspect item state Then use Replace H D Middle aspect words As the hidden state, we get the initial context hidden state sequence 3. According to claim 1, a fine-grained sentiment analysis method based on denoising template and abstract semantic representation information is characterized in that: In the step 2, the specific method is: AMR replaces the traditional syntactic structure dependency tree, extracts the semantic information contained in the context, explicitly represents the propositional structure of the sentence, removes information in the text that is irrelevant to the semantic task, and highlights important information; Firstly, the AMR graph is parsed from the input sentence using the off-the-shelf parser SPRING. Secondly, the AMR is aligned using the aligner LEAMR. Based on the alignment, the AMR relationship between words in the sentence is reconstructed to obtain the converted AMR with words as nodes.

4. According to claim 1, a fine-grained sentiment analysis method based on denoising template and abstract semantic representation information is characterized in that: The specific method in step 3 is: After alignment, we need to obtain the embedding of AMR information. For the words in the sentence, we also use BERT as the encoder to obtain the context embedding H. A ={h1, h2, .., h n }, for the edges in AMR, the relationship between nodes is represented as an adjacency matrix R = {r ij |1≤i,j≤n}, where r ij It is the word w i and word w j The embedding of the edge label between them, if in AMR w i and w j If there is no edge between them, then give r ij Assign an embedding represented as "none", and the edge label embedding is also obtained from the pre-trained model; global features are realized by broadcasting local information along the path in the graph, integrating information from AMR and sentences to obtain optimized relational features. It is necessary to add a linear activation layer to aggregate semantic related information, as shown in the following formulas (1)(2)(3)(4)(5): r′ ij =LayerNorm(r ij ) (1) g ij =sigmoid(Linear(r′ ij )) (2) a ij =g ij ⊙Linear(r′ ij ) (3) where r′ ij Represents r ij The standardized output, g ij Represents the output obtained by the activation function sigmoid, a ij Indicates the output of the inner product of the obtained results. represents the output obtained through the linear layer and residual connection; Finally, the aggregate R is transformed into F The output is converted into a relational attention weight matrix Used for subsequent calculations.

5. According to claim 1, a fine-grained sentiment analysis method based on denoising template and abstract semantic representation information is characterized in that: The specific method in step 4 is: The embedding code calculates the attention weight through the classic self-attention mechanism module formula (6): Where Q and K are d-dimensional input vectors, W Q and W K Yes and R d×d The learnable weights of the same size are used to construct the self-attention mechanism for relational enhancement. The relational attention weight matrix R in AMR is F Added to the original attention weight, it is expressed as formula (7): Join R F , the attention output is further guided by the semantic information from AMR, thereby improving the effective attention to semantic keywords. Finally, the relation-enhanced aspect item representation matrix is ​​obtained For subsequent classification.

6. According to claim 1, a fine-grained sentiment analysis method based on denoising template and abstract semantic representation information is characterized in that: In step 5, the model is trained and the probability of mapping to three sentiment polarities is calculated through a fully connected softmax layer as formula (8): Using cross entropy loss as the objective function is formula (9): Where y is the true sentiment polarity, D contains all matching pairs of aspect items and sentiment items, and C contains all sentiment polarities.

Images