Aspect-level sentiment analysis method based on task characteristic prompt and multi-level language characteristic enhancement
By adopting the method of task feature prompting and multi-level language feature enhancement in aspect-level sentiment analysis, the problems of traditional fine-tuning models in computing resources, forgetting and labeling data dependencies are solved, and more efficient and robust model performance is achieved.
Patent Information
- Application Number
- CN202510059545.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-15
- Publication Date
- 2025-05-13
AI Technical Summary
Traditional fine-tuning pre-trained models have problems such as high computing resource requirements, catastrophic forgetting and strong dependence on labeled data in aspect-level sentiment analysis.
Using a method based on task feature prompting and multi-level language feature enhancement, the model's performance in aspect-level sentiment analysis is improved by designing task-specific prompt templates and multi-level language feature enhancement modules, combining prompt learning and language feature enhancement.
Effectively reduces computing resource consumption, avoids catastrophic forgetting, reduces dependence on labeled data, and improves the model's understanding ability and performance in complex tasks.
Smart Images
Figure CN119990138A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of natural language processing of computer technology, and in particular to an aspect-level sentiment analysis method based on task characteristic prompts and multi-level language feature reinforcement. Background Art
[0002] In today's rapidly developing Internet era, there is a lot of user language information. Mining the hidden user attributes behind these sentences is an important and meaningful subtask of natural language processing. For example, by mining the information behind user comments, businesses can more accurately identify the advantages and disadvantages of products or services, thereby improving and enhancing user satisfaction; at the same time, companies can monitor public opinion, brand management and crisis public relations in real time. Aspect-sentiment triples are designed to extract aspect entities and corresponding opinion entities from sentences and predict sentiment polarity. They combine multiple NLP tasks such as sentiment analysis, aspect extraction, and sentiment polarity detection.
[0003] Pre-trained models have become a popular choice in various fields because they can learn a wide range of language features and other knowledge on large-scale datasets and can be adapted to different downstream tasks through fine-tuning. However, there are three problems with traditional fine-tuning: 1) high computing resource requirements; 2) catastrophic forgetting; 3) dependence on a large amount of labeled data. Prompted learning can effectively solve these problems. However, prompt learning has been proven to be an effective solution that can reduce computing resource consumption, avoid catastrophic forgetting, and reduce dependence on labeled data while still maintaining the efficiency of the model. Initial research focused on using fixed prompt templates to guide pre-trained models for downstream task training. These templates are simple and general, and can achieve good results in many tasks. Some studies related to aspect-level sentiment also borrowed this approach and believed that training the model through prompt templates can bring improvements. Traditional prompt templates have limited effect in handling these complex tasks, so it is necessary to design more sophisticated prompt templates that fit the characteristics of the task to guide the model to effectively complete tasks such as aspect-level sentiment analysis.
[0004] In addition to the widespread application of pre-trained models and prompt learning in ABSA tasks, constructing dependencies between words based on language features has also become an important research direction in this field. Since this method directly acts on the relationship between words, many researchers have begun to focus on using graph structures to achieve feature extraction and information mining. Typical models include GCN, GraphSAGE, and GAT. However, graph structure models face challenges in applications, such as over-smoothing, feature capture limitations, and insufficient expression of complex relationships. These problems have weakened the expressiveness of the model in the ASTE task to a certain extent. To this end, it is necessary to strengthen the language features to improve the graph model's ability to capture the dependencies between nodes and key information in the graph structure, thereby improving the overall performance of the model. Summary of the invention
[0005] The purpose of the present invention is to provide an aspect-level sentiment analysis method based on task-specific prompts and multi-level language feature enhancement, which adopts task-specific prompts and multi-level language feature enhancement models to solve the ASTE task by combining prompt learning and language feature enhancement. Inspired by traditional prompt learning, a prompt template with task characteristics is designed to enable the model to process the ASTE task more efficiently. On the other hand, by enhancing multiple language features from multiple levels, the model can more comprehensively capture and mine the deep information in the language.
[0006] The above technical objectives of the present invention are achieved through the following technical solutions: an aspect-level sentiment analysis method based on task characteristic prompts and multi-level language feature enhancement, the specific steps of which are as follows:
[0007] A. BERT jointly encodes the input raw text and the prompts with task characteristics, and constructs language features together with the text and the prompts;
[0008] B. The multi-level language feature enhancement module enhances the reconstructed language features from the global and local levels respectively, and integrates the multi-level language features through the attention mechanism;
[0009] C. Use double affine transformation and graph convolutional network for further processing, decoding and prediction.
[0010] Preferably, the BERT in step A above jointly encodes the input original text and the prompt with task characteristics, and constructs language features together with the text and the prompt. The specific steps are as follows:
[0011] A1. Define a prompt template T with task characteristics and input it together with the original text as I=S+T;
[0012] A2. Use BERT as the text encoder to jointly encode the input to obtain a preliminary feature representation:
[0013]
[0014] A3. Obtain the language feature relationship between each pair of words with the help of dependency parser and represent it in the form of adjacency matrix:
[0015] R k =f(H);
[0016] Here k represents four language features, namely psc, sdt, tbd, and rpd.
[0017] Preferably, the multi-level language feature enhancement module in the above step B enhances the reconstructed language features from the global and local levels respectively, and fuses the multi-level language features through the attention mechanism, and the specific steps are as follows:
[0018] is the joint weight matrix, A row and A col Dynamically generate directional attention weights, R row , R col and R glob Representation of linguistic features representing different directions;
[0019] B2. For global level language feature enhancement, first generate a comprehensive feature representation with direction awareness:
[0020] R row =M T R,
[0021] R col =R·M,
[0022]
[0023] B3. Perform dynamic feature fusion on the comprehensive feature representation to obtain global level language enhancement R1:
[0024] R1=A row ⊙R row +A col ⊙R col +X glob ;
[0025] B4. For local level language feature enhancement, detect feature changes of adjacent word pairs and use these changes as semantic boundary features:
[0026] D = R - Maxpool (R),
[0027] R2=R+σ(Norm(Conv(D)));
[0028] B5. Use the attention mechanism to integrate the language features after multi-level language enhancement:
[0029]
[0030] Preferably, the above step C uses double affine transformation and graph convolutional network for further processing, decoding and prediction, and the specific steps are as follows:
[0031] C1, h i and h j Represents a word pair, Represents the comprehensive features after the interaction of different language features;
[0032] C2. Calculate the correlation strength between word pairs of different language features through double affine transformation to generate a correlation matrix. Then, integrate the correlation matrix into the language features and use GCN aggregation to achieve interaction and integration of feature information:
[0033] B=Biaffine(h i ,h j ),
[0034]
[0035] C3. Combine the word point information with the enhanced language features to form a word pair feature expression, and input it into the fully connected layer to obtain the label possibility distribution:
[0036]
[0037] p ij =softmax(W p x ij +b p ).
[0038] In summary, the present invention has the following beneficial effects:
[0039] The aspect-level sentiment analysis method based on task characteristic prompts and multi-level language feature enhancement of the present invention is more challenging when the complexity of the text increases. While prompt learning is added, a more appropriate method is also needed to re-stimulate the guidance of the prompt template. Therefore, a prompt template with task characteristics is designed and a corresponding feature enhancement module is designed to fully stimulate the performance of the template.
[0040] The present invention takes into account that local features focus on the relationship between local information such as sentiment words, aspect words and modifiers, while global features focus on the grammatical structure of sentences and the relationship between multiple sentiment targets in long texts. A multi-level language feature enhancer is designed. This enhancer not only promotes the effective complementarity of language features, but also enhances the performance of the model in specific tasks by introducing prompt templates. At the same time, we dynamically fuse different language features through the attention mechanism, so that the model can adaptively adjust the importance of each feature in the task.
[0041] The method of the present invention was experimented on four public datasets, using F1 as the evaluation indicator to compare 14 models. The results showed that it performed better than existing models in multiple benchmark tests. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 It is an explanation of the four language features used in the present invention;
[0043] Figure 2 It is a model result diagram of an aspect-level sentiment analysis method based on task characteristic prompts and multi-level language feature enhancement of the present invention;
[0044] Figure 3 It is the process of strengthening the global hierarchical features described in step B1 and step B2;
[0045] Figure 4 It is the process of strengthening the local level features described in step B3;
[0046] Figure 5 It is the result of the aspect-level sentiment analysis method based on task characteristic prompts and multi-level language feature enhancement on the data set. DETAILED DESCRIPTION
[0047] The present invention is further described in detail below in conjunction with the accompanying drawings.
[0048] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.
[0049] Reference Figure 2 The aspect-level sentiment analysis method based on task characteristic prompts and multi-level language feature enhancement of the present invention comprises the following steps:
[0050] Step A: BERT jointly encodes the input original text and the prompt with task characteristics, and constructs language features together with the text and the prompt. Figure 2 , the specific steps are as follows:
[0051] Step A1, define a prompt template T with task characteristics, and input it together with the original text as I=S+T;
[0052] Step A2: Use BERT as the text encoder to jointly encode the input and obtain a preliminary feature representation:
[0053]
[0054] Step A3: Obtain the language feature relationship between each pair of words with the help of a dependency parser and represent it in the form of an adjacency matrix:
[0055] R k =f(H).
[0056] Step B: The multi-level language feature enhancement module enhances the reconstructed language features from the global and local levels respectively, and fuses the multi-level language features through the attention mechanism. Figure 2 , Figure 3 and Figure 4 , the specific steps are as follows:
[0057] Step B1: For global level language feature enhancement, first generate a comprehensive feature representation with direction awareness:
[0058] R row =M T R,
[0059] R col =R·M,
[0060]
[0061] Step B2: Perform dynamic feature fusion on the comprehensive feature representation to obtain global level language enhancement R1:
[0062] R1=A row ⊙R row +A col ⊙R col +X glob ;
[0063] Step B3: For local level language feature enhancement, detect feature changes of adjacent word pairs and use these changes as semantic boundary features:
[0064]
[0065] R2=R+σ(Norm(Conv(D)));
[0066] Step B4: Use the attention mechanism to fuse the language features after multi-level language enhancement:
[0067]
[0068] Step C: Use double affine transformation and graph convolutional network for further processing, decoding and prediction, refer to Figure 2 , the specific steps are as follows:
[0069] Step C1: Calculate the correlation strength between word pairs of different language features through double affine transformation to generate a correlation matrix. Then, integrate the correlation matrix into the language features and use GCN aggregation to achieve interaction and integration of feature information:
[0070] B=Biaffine(h i ,h j ),
[0071]
[0072] Step C2: Combine the word point information with the enhanced language features to form a word pair feature expression, and input it into the fully connected layer to obtain the label possibility distribution:
[0073]
[0074] p ij =softmax(W p x ij +b p ).
[0075] Fourteen methods are compared on four public datasets, including CLMA+, RINANTE+, Li-unified-R, TSF, OTE-MTL, BMRC, Span-ASTE, DE-OTE-BISDD, CopyMTL, EMC-GCN, DGEIAN, ESGAT, SBRS, and SA-Transformer. The experiments are tested on four public datasets: Res14, lap14, Res15, and Res16, and the indicator is F1.
[0076] The experimental results show that the present invention shows good results on all data sets, and the following facts can be found:
[0077] 1. Compared with the pipeline model, this method effectively avoids error propagation through an end-to-end architecture and can fully integrate local and global features;
[0078] 2. Although the multi-task learning model can handle multiple subtasks at the same time, it has limitations in capturing fine-grained semantic relationships within sentences. The task-specific prompts and multi-level language feature enhancement of this method effectively make up for this deficiency;
[0079] 3. Compared with graph neural network methods, although graph neural networks perform well in modeling global dependencies, they may encounter challenges in dealing with overlapping triplets. The bidirectional preservation mechanism of this method ensures robust handling of complex label structures.
[0080] The following analysis is made: This method simplifies the formalization process of the task by introducing task-specific prompts and strengthens semantic guidance, so that the model can focus more on the aspect-viewpoint-sentiment relationship. Its multi-level feature enhancement module effectively combines syntactic, semantic and positional information, greatly improving the ability to extract overlapping triples and complex relations. These advantages make this method not only a state-of-the-art model for aspect-sentiment triple extraction, but also a general solution that adapts to different data sets and task complexities.
Claims
1. An aspect-level sentiment analysis method based on task characteristic prompts and multi-level language feature enhancement, characterized in that: The specific steps are as follows: A. BERT jointly encodes the input raw text and the prompts with task characteristics, and constructs language features together with the text and the prompts; B. The multi-level language feature enhancement module enhances the reconstructed language features from the global and local levels respectively, and integrates the multi-level language features through the attention mechanism; C. Use double affine transformation and graph convolutional network for further processing, decoding and prediction.
2. The aspect-level sentiment analysis method based on task characteristic prompts and multi-level language feature enhancement according to claim 1 is characterized in that: In step A above, BERT jointly encodes the input original text and the prompt with task characteristics, and constructs language features together with the text and the prompt. The specific steps are as follows: A1. Define a prompt template T with task characteristics and input it together with the original text as I=S+T; A2. Use BERT as the text encoder to jointly encode the input to obtain a preliminary feature representation: A3. Obtain the language feature relationship between each pair of words with the help of dependency parser and represent it in the form of adjacency matrix: R k =f(H); Here k represents four language features, namely psc, sdt, tbd, and rpd.
3. The aspect-level sentiment analysis method based on task characteristic prompts and multi-level language feature enhancement according to claim 1 is characterized in that: The multi-level language feature enhancement module in the above step B enhances the reconstructed language features from the global and local levels respectively, and fuses the multi-level language features through the attention mechanism. The specific steps are as follows: B1. is the joint weight matrix, A row and A col Dynamically generate directional attention weights, R row , R col and R glob Representation of linguistic features in different directions; B2. For global level language feature enhancement, first generate a comprehensive feature representation with direction awareness: R row =M T ·R, R col =R·M, B3. Perform dynamic feature fusion on the comprehensive feature representation to obtain global level language enhancement R1: R1=A row ⊙R row +A col ⊙R col +X glob ; B4. For local level language feature enhancement, detect feature changes of adjacent word pairs and use these changes as semantic boundary features: D = R - Maxpool (R), R2=R+σ(Norm(Conv(D))); B5. Use the attention mechanism to integrate the language features after multi-level language enhancement:
4. The aspect-level sentiment analysis method based on task characteristic prompts and multi-level language feature enhancement according to claim 1 is characterized in that: The above step C uses double affine transformation and graph convolutional network for further processing, decoding and prediction. The specific steps are as follows: C1, h i and h j Represents a word pair, Represents the comprehensive features after the interaction of different language features; C2. Calculate the correlation strength between word pairs of different language features through double affine transformation to generate a correlation matrix. Then, integrate the correlation matrix into the language features and use GCN aggregation to achieve interaction and integration of feature information: B=Biaffine(h i ,h j ), C3. Combine the word point information with the enhanced language features to form a word pair feature expression, and input it into the fully connected layer to obtain the label possibility distribution: p ij =softmax(W p x ij +b p )。