Aspect category sentiment analysis method based on prompt prediction and semantic enhancement

Abstract

The invention provides an aspect category sentiment analysis method based on prompt prediction and semantic enhancement, and relates to the technical field of natural language process.The method comprises the steps that an original text is input into an autoregression module to generate an extended text, and the extended text and the original text are spliced to form an enhanced text; constructing an input sequence by the emotion polarity label, the enhanced text and the prompt template according to a predefined format; an input sequence is encoded through the RoBERTa encoder module to obtain context vector representation; inputting the context vector representation into an aspect category classification module, extracting an information vector through an attention function, mapping the information vector into probability distribution of label words, and obtaining an aspect category prediction result; inputting the context vector sequence into a sentiment classification module, obtaining bidirectional LSTM representation through BiLSTM, and obtaining a sentiment polarity classification result through feature fusion; and integrating the aspect category prediction result and the emotion polarity classification result, and outputting a prediction result. According to the invention, the accuracy of aspect category sentiment analysis is improved.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to an aspect category sentiment analysis method based on cue prediction and semantic enhancement. Background Technology

[0002] Currently, Category Aspect Sentiment Analysis (ACSA), as a core task of fine-grained sentiment analysis in natural language processing, has undergone technological development. Its evolution has progressed from relying on Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Long Short-Term Memory Networks (LSTMs) as basic feature extractors, to introducing attention functions to better capture the semantic associations between contextual words and specific aspects, and then to utilizing pre-trained language models (such as BERT) combined with BiLSTM (Bidirectional Long Short-Term Memory Network), Graph Convolutional Networks (GCNs), and other models to obtain more comprehensive contextual representations. In recent years, cue learning (PL) has gradually emerged. It reconstructs downstream tasks into mask prediction tasks in the pre-training stage by designing templates, aiming to more efficiently stimulate the knowledge in the pre-trained model and reduce dependence on labeled data.

[0003] Despite continuous technological advancements, the ACSA task still faces three core bottlenecks in practical applications: First, current high-performance models heavily rely on large-scale, high-quality labeled data, leading to high data costs. Meanwhile, cue learning methods, designed to reduce data dependence, are limited by the masking language modeling capabilities of the pre-trained models themselves. Second, short texts such as online comments generally exhibit sparse vocabulary and loose structure, creating inherent "semantic gaps" that make it difficult for models to capture unexpressed potential emotional information, thus hindering analytical accuracy. Furthermore, traditional network structures often struggle to effectively coordinate global context and local dependencies when processing complex texts, resulting in insufficient sensitivity to subtle local changes or neglect of distant contextual constraints, impacting the deep understanding of complex sentence structures and emotions.

[0004] Therefore, there is an urgent need for a new method that can effectively reduce data dependence and enhance the ability to represent the deep semantics of short texts, thereby improving the performance of aspect category sentiment analysis in terms of data efficiency, semantic understanding and sentiment judgment accuracy. Summary of the Invention

[0005] The purpose of this invention is to provide an aspect category sentiment analysis method based on cue prediction and semantic enhancement, which overcomes the problems of existing technologies such as high dependence on large-scale labeled data, insufficient semantic information in short texts, and poor coordination between global and local semantic representations. It can achieve efficient and accurate aspect category identification and sentiment polarity judgment, enhance the ability to understand the deep semantics of short texts, improve adaptability in data-scarce scenarios, and improve the accuracy and robustness of aspect category sentiment analysis.

[0006] To achieve the above objectives, this invention provides an aspect category sentiment analysis method based on cue prediction and semantic enhancement, comprising the following steps: Step S1: Input the original text into the autoregressive module to generate extended text, and then concatenate the extended text with the original text to form enhanced text; Step S2: Construct the input sequence by combining the sentiment polarity labels, enhanced text, and cue templates according to a predefined format; Step S3: The input sequence is encoded by the RoBERTa encoder module to obtain a context vector representation; Step S4: Input the context vector into the aspect category classification module, extract the information vector through the attention function, map it to the probability distribution of the label words, and obtain the aspect category prediction result; Step S5: Input the context vector sequence into the sentiment classification module, obtain a bidirectional LSTM representation through BiLSTM, and obtain the sentiment polarity classification result through feature fusion; Step S6: Based on the aspect category prediction results and the sentiment polarity classification results, integrate and output the prediction results.

[0007] Preferably, in step S1, the autoregressive module is a T5 autoregressive generative model.

[0008] Preferably, in step S2, the prompt template uses the following fixed sentence structure: This sentence mainly describes... <mask>.

[0009] Preferably, in step S2, the predefined format is: <s>Emotional polarity tags< / s> Original text + extended text hint template.

[0010] Preferably, in step S2, three input sequences are constructed, with their sentiment polarity labels corresponding to positive, neutral, and negative, respectively.

[0011] Preferably, step S4 specifically includes: The context vector representation is used as the input aspect category classification module, and the full-text vector representation is extracted and... <mask>The labeled vector is used to calculate the association weight between the two through an attention function, and the aspect-related information vector is extracted. Connect information vectors with <mask>The label vectors are concatenated and dimensionality reduced to obtain the overall predicted label word vectors; The label word vectors are mapped to a pre-created vocabulary through a RoBERTa pre-trained linear layer, which is then converted into a probability distribution of all words in the vocabulary. Based on probability distribution, the probability of the tag words corresponding to each aspect category is statistically analyzed, and the scores corresponding to each tag are calculated and compared to obtain the aspect category prediction results.

[0012] Preferably, in step S5, feature fusion specifically involves: extracting the first label from the context vector representation. <s>The vector representation is then concatenated and fused with the bidirectional LSTM representation to form a fused feature vector.

[0013] Preferably, step S5 specifically includes: The context vector sequence is input into the sentiment classification module, and the last hidden state of the forward LSTM and the first hidden state of the backward LSTM are obtained through BiLSTM respectively. The last hidden state of the forward LSTM and the first hidden state of the backward LSTM are concatenated to obtain the bidirectional LSTM representation. Perform feature fusion: Extract the first label from the context vector representation. <s>The vector representation is then concatenated and fused with the bidirectional LSTM representation to form a fused feature vector. The fused feature vectors are passed through a fully connected layer and a softmax activation function to output the sentiment polarity classification result.

[0014] Preferably, the method is executed by a PRSM model that integrates an autoregressive module, a RoBERTa encoder module, an aspect category classification module, and a sentiment classification module.

[0015] Preferably, the PRSM model is optimized through end-to-end joint training, with the total loss function... Loss function based on aspect category prediction And sentiment classification loss function Together they constitute, and the specific expression is: ; ; ; in, For real labels, For the number of aspect categories, For the first The categories in the first Predicted probabilities on sentiment polarity labels As a label for the true polarity of emotions, For the first The predicted probability of each sentiment polarity label for The regularization coefficient of regularization, These are all learnable parameters in the PRSM model.

[0016] Therefore, the present invention employs the aforementioned aspect category sentiment analysis method based on cue prediction and semantic enhancement, and the beneficial technical effects are as follows: (1) This invention reconstructs the aspect category prediction task into a mask language model task by constructing a specific prompt template, and utilizes the mask prediction mechanism of the RoBERTa pre-trained language model to achieve effective transfer from pre-trained knowledge to downstream tasks, effectively reducing the dependence on large-scale labeled data and improving the model's adaptability and generalization ability in small sample scenarios.

[0017] (2) This invention utilizes the T5 autoregressive generative model to semantically expand short texts, effectively filling the semantic gaps in short texts, enhancing the ability to capture implicit emotional information, and solving the problems of semantic sparsity and difficulty in capturing implicit information in short texts.

[0018] (3) This invention integrates local dependencies and global context through the sentiment classification module in conjunction with RoBERTa’s global semantic representation and BiLSTM’s sequence representation, and generates more discriminative sentiment feature representations by splicing and nonlinear transformation, thereby achieving a deep understanding of the multi-granular semantics of complex texts and improving the accuracy of sentiment polarity judgment.

[0019] (4) This invention integrates aspect category classification and sentiment classification into a unified framework for collaborative training through an end-to-end optimization method of multi-task joint training. By jointly optimizing the loss functions of the two sub-tasks, the model can learn and capture the inherent semantic relationship between aspects and sentiment simultaneously, effectively promoting knowledge sharing and information complementarity between the two sub-tasks. This avoids the error accumulation problem caused by stage separation in traditional pipeline methods, thereby significantly improving the accuracy and robustness of aspect category sentiment analysis as a whole. Attached Figure Description

[0020] Figure 1 This is a flowchart of an aspect category sentiment analysis method based on cue prediction and semantic enhancement according to the present invention; Figure 2 Extended text example generated for the T5 autoregressive generative model; Figure 3 This is a schematic diagram of the PRSM model. Detailed Implementation

[0021] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0022] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0023] Example 1 This embodiment provides an aspect category sentiment analysis method based on cue prediction and semantic enhancement. The method is executed by a PRSM model, such as... Figure 3 As shown, the PRSM model is an end-to-end deep learning framework that integrates an autoregressive module, a RoBERTa (a robustly optimized BERT pre-training method) encoder module, an aspect category classification module, and a sentiment classification module. Through the synergy of these modules, the PRSM model divides the ACSA task into two closely related sub-tasks: aspect category prediction and sentiment polarity prediction. Specifically: For a sentence or a sequence of text from a dataset, as input, it can be represented as: ; in, The number of words in the text. For text, i.e., the input sequence. For the first in the text One word.

[0024] For a set of predefined aspect categories, it can be represented as: ; in, For the predicted aspect category, Let m be the predicted aspect category.

[0025] Generally, an aspect category consists of two parts: an entity and an attribute. Taking "food#quality" as an example, "food" represents the entity, while "quality" represents the attribute corresponding to the entity. In this embodiment, one aspect category corresponds to one tag set. The tag set used is predefined and designed based on semantic analysis of the domain text or common usages. A tag set contains multiple tag words, the selection of which aims to cover the typical expressions of that aspect category to ensure semantic relevance, as shown in Table 1. Therefore, each tag can be represented as: ; in, The number of tag words corresponding to each tag. For the first The first aspect category corresponds to the Each tag word.

[0026] Table 1 Examples of tag words

[0027] Emotional polarity is generally categorized into three types: positive, neutral, and negative, expressed as follows: ; in, To be positive, To remain neutral For negative reasons, .

[0028] Therefore, given a text S, finding its corresponding label (i.e., aspect category) through the label words can be achieved using an aspect category prediction function. To represent, the specific expression is: ; in, These are the tag words extracted from the text S.

[0029] Then, based on the identified aspect categories, their corresponding sentiment polarity is further determined. This process can be achieved using a sentiment classification function. To represent. Sentiment classification function. It can measure the sentiment probability distribution corresponding to the identified aspect categories in the text, and the specific expression is: ; like Figure 1 As shown, an aspect category sentiment analysis method based on cue prediction and semantic enhancement includes the following steps: Step S1: Input the original text into the autoregressive module to generate extended text, and then concatenate the extended text with the original text to form enhanced text.

[0030] To alleviate the inherent semantic sparsity problem of short texts and thus improve the accuracy of the PRSM model in understanding implicit viewpoints, this embodiment introduces the T5 (Text-to-Text Transfer Transformer) autoregressive generative model as an autoregressive module to semantically expand the input raw text and generate expanded text. The core idea of ​​the T5 autoregressive generative model is to unify all natural language processing tasks into text generation problems, which is highly consistent with the requirement of generating text-related explanatory content in this embodiment. The specific expression is as follows: ; in, The original text input. For the expanded text output, For the generated first One word, For the already generated previous One word, The length of the generated text, This is the logarithmic probability.

[0031] To ensure the T5 autoregressive generative model can correctly perform the semantic expansion task, it needs to be preprocessed before the original text is input. Specifically, a system prompt text is added before the original text. Therefore, the final input format for the T5 autoregressive generative model is "system prompt text + original text". The system prompt text aims to guide the T5 autoregressive generative model to perform deep semantic expansion of the original text, and the expanded content must be strictly based on aspects covered by the original text.

[0032] The system prompt text reads: "You are an interpretive assistant. Your task is to expand upon and explain how specific aspects mentioned in the input text affect the user experience. During the expansion, you should define the scope of content based on the provided tags, ensuring that each area (e.g., laptops, restaurants, etc.) is accurately described. When expanding, focus on the specific aspects mentioned in the original text. For example, if 'battery' is mentioned, explore battery performance in depth; if 'restaurant service' is mentioned, analyze the impact of service quality on the experience. Each aspect's expansion should provide specific technical or practical details explaining its role and impact on the user experience. For example, in the laptop field, discuss battery performance, screen quality, or CPU speed in detail, etc." How does it affect the overall user experience? In the restaurant sector, the discussion should focus on how food quality, service level, or price influences customer satisfaction and the decision-making process. Expanded content should be kept between 50 and 200 words, maintaining a natural flow, avoiding the introduction of content not mentioned in the original text, and aiming to deepen understanding of each aspect rather than adding irrelevant information. Emotional consistency is crucial; if the original text conveys a negative tone, it should be maintained in the expanded content; if the original text is neutral or positive, consistency should be maintained. The expanded content should not deviate from the aspects mentioned in the original text and should avoid including content unrelated to the original. The focus should be on in-depth analysis of the impact of each aspect on the user experience, rather than expanding to irrelevant or vague content, ensuring that all information is closely related and effectively enhances the user's understanding of each aspect. The T5 autoregressive generative model performs deep semantic expansion on the original text, covering only the specific aspects involved in the original text. To systematically guide and constrain the quality and relevance of the expanded text during the generation stage, this embodiment employs a bundle search-based decoding strategy. By adjusting parameters such as length reward and repetition penalty, the coherence, information density, and expressive diversity of the generated content are ensured. Furthermore, the generated expanded text does not directly provide ACSA prediction results but instead generates explanatory content related to the original text, indirectly enriching the semantics by supplementing relevant vocabulary and context.

[0033] like Figure 2 As shown. For example, the original text indicates that the user's sentiment polarity towards the aspect category "battery quality" is negative. Although the extended text generated by the T5 autoregressive generative model does not directly provide the prediction results of the aspect category (ACD) or sentiment polarity (SC), the extended text indirectly reflects the computer's practicality through words such as "battery performance," "smooth," "limited use," and "long-term use," while "reduced reliability" reflects a negative sentiment polarity from another perspective.

[0034] Meanwhile, to avoid the inherent "illusion" and semantic contradiction risks of the T5 autoregressive generative model when generating extended text, the generated extended text must undergo rigorous data cleaning and semantic rationality verification before being concatenated with the original text: only high-quality extended text that passes the verification is retained as auxiliary context, while unreasonable generated content is directly discarded, in order to ensure the absolute dominance of the original input semantics and make the semantic enhancement process highly robust.

[0035] Finally, the extended text is concatenated with the original text to form enhanced text, which is then input into the subsequent RoBERTa encoder module as supplementary contextual information. This fills in the information gaps in the short text and enriches the overall semantic representation.

[0036] Step S2: Construct the input sequence by combining the sentiment polarity labels, enhanced text (i.e., original text + extended text), and cue templates in a predefined format.

[0037] In this embodiment, the input sequence They are combined into the following forms according to a predefined format: S= <s> Emotional polarity tags< / s> Original text + expanded text< / s> Prompt Template< / s> .

[0038] in, <s> and< / s> These are special markers designed to help the PRSM model recognize the entire input sequence and perform correct contextual understanding.

[0039] The prompt template uses "This sentence mainly describes..." <mask>The fixed sentence structure.

[0040] Since sentiment polarity labels include three types: positive, negative, and neutral, each original text will be expanded into three parallel input sequences when constructing the input sequence. Each input sequence will have a different sentiment polarity label: positive, neutral, and negative, while the rest will remain unchanged.

[0041] To more clearly and completely illustrate the technical solution of this embodiment, the following is combined with... Figure 3 Let's illustrate this with a specific example.

[0042] Let's say the original text input is: "The burger was pretty good, but I don't expect them to provide extra napkins." After inputting the original text into the autoregressive module (i.e., the T5 autoregressive generative model), it combines the system prompts to generate expanded text. The expanded text might focus on "food quality" and "service," for example, "The burger patty was juicy and the bun was fluffy" reflects good food quality. However, "The restaurant was stingy and inattentive in providing details like extra napkins" reflects the service experience.

[0043] Subsequently, the enhanced text (i.e., original text + expanded text), sentiment polarity labels, and cue templates are constructed into three input sequences, specifically: ①〈s〉Positive〈s〉Hamburger…napkin + extended text〈s〉This sentence mainly describes what <mask>〈 / s〉.

[0044] ②〈s〉Neutral 〈s〉Hamburger…napkin + extended text 〈s〉This sentence mainly describes <mask>〈 / s〉.

[0045] ③ <s>Negative< / s> Hamburger…napkin + extended text< / s> This sentence mainly describes what <mask>〈 / s〉.

[0046] Step S3: The input sequence is encoded by the RoBERTa encoder module to obtain a context vector representation.

[0047] This embodiment uses the RoBERTa encoder module to process the input sequence. The code encodes the context features, converting them into a high-dimensional context vector representation. The RoBERTa encoder module uses its embedded RoBERTa pre-trained language model to extract context features, optimizing the processing of prompt templates. <mask>Predicting masked words. The specific expression is: ; in, Represented as a context vector, , The dimension of the hidden layer of the RoBERTa pre-trained language model. For the RoBERTa encoder function, It is the set of real numbers.

[0048] Step S4: The context vector is input to the aspect category classification module. An information vector is extracted using an attention function and mapped to the probability distribution of the label words to obtain the aspect category prediction result. Specifically, this includes: The context vector representation is used as the input aspect category classification module, and the full-text vector representation is extracted. and <mask>Tag vector representation And through attention function Calculate the association weight between the two and extract the aspect-related information vector. , The specific expression is: ; ; in, , , For the first Full-text vector representation of 1 word, , , .

[0049] Information vector and <mask>Tag vector representation By concatenating and reducing the dimensionality, we obtain the overall predicted label word vector. , The specific expression is: ; in, For vector concatenation, It is a fully connected layer used to transform vectors from high dimensions to low dimensions.

[0050] Tag word vectors The RoBERTa pre-trained language model is mapped to a predefined vocabulary through linear layers, transforming it into a probability distribution of all words in the vocabulary. The specific expression is as follows: ; in, The probability of a word in the vocabulary. , , For learnable parameters, , The number of words in the vocabulary list. This is the weight matrix of the linear layer.

[0051] Based on probability distribution, the corresponding tag words for each aspect category are statistically analyzed. The probability of each tag is calculated, and the scores corresponding to each tag are compared to obtain the aspect category prediction results. , . For the first Each tag word The specific expression is: ; ; in, For the first The probability distribution of each tag word , The length of the tag word, To extract tag words from the vocabulary, For parameter matrices, , Softmax is the activation function used to concatenate the scores. For index variables.

[0052] In the processing of the "hamburger" example sentence, the aspect category classification module generates differentiated predictions for input sequences with different sentiment polarities: exist <s> positive< / s> In the corresponding input sequence, the PRSM model focuses on the positive information in the original text, thereby... <mask>Location has a high probability of predicting tags related to "food quality" (such as "burger", "delicious" etc.).

[0053] exist <s> negative< / s> In the corresponding input sequence, the PRSM model focuses on negative information in the original text, thereby... <mask>Location has a high probability of predicting tags related to "service#regular" (such as "service", "napkins" etc.).

[0054] exist <s> neutral< / s> In the corresponding input sequence, since there is no truly neutral sentiment object in the original text, the PRSM model cannot find a clear and high-probability aspect category to correspond to it, therefore its performance... <mask>The probability distribution calculated for all predefined tag words is relatively uniform, lacking a significant high probability peak.

[0055] Step S5: The context vector sequence is also input into the sentiment classification module. A bidirectional LSTM representation is obtained through BiLSTM, and the sentiment polarity classification result is obtained through feature fusion. Specifically, this includes: The context vector sequence is input into the sentiment classification module, and the final hidden state of the forward LSTM is obtained through BiLSTM. and the first hidden state of the backward LSTM The specific expression is: ; ; The last hidden state of the forward LSTM and the first hidden state of the backward LSTM are concatenated and merged to obtain the bidirectional LSTM representation. The specific expression is: ; Perform feature fusion: Extract the first label from the context vector representation. <s>vector representation , and bidirectional LSTM representation The features are spliced ​​and merged to form a fused feature vector. The specific expression is: ; ; in, The weight matrix of the linear transformation layer. , The bias vector of the linear transformation layer. , For random inactivation, It is a linear rectified function. For layer normalization, This represents the concatenated feature vector. The fused feature vector is then passed through a fully connected (FC) layer and a softmax activation function to output the sentiment polarity classification result. The specific expression is: ; in, For the prediction results of sentiment polarity, , , For trainable parameters, , .

[0056] For the example sentence "hamburger", the sentiment classification module will evaluate each of the three input sequences separately. The PRSM model is likely to be... <s> positive< / s> The corresponding input sequence is used to calculate a high-probability "positive" sentiment. <s> negative< / s> The corresponding input sequence is used to calculate a high probability of "negative" sentiment, while... <s> neutral< / s> The corresponding input sequence is used to calculate the low probability.

[0057] The PRSM model is optimized through end-to-end joint training. During training, the total loss function... Loss function based on aspect category prediction And sentiment classification loss function Together they constitute.

[0058] When performing aspect class prediction for multi-label classification problems, the aspect class prediction loss function is... The specific expression is: ; in, For real labels, For the number of aspect categories, For the first The categories in the first Predicted probabilities on sentiment polarity labels.

[0059] When performing sentiment polarity prediction, the sentiment classification loss function The specific expression for calculating binary cross-entropy is as follows: ; in, As a label for the true polarity of emotions, For the first The predicted probability of an emotional polarity label.

[0060] To improve the training performance of the PRSM model, mini-batch stochastic gradient descent is used to jointly optimize the aspect class prediction loss function and the sentiment classification loss function, minimizing their sum, which is the total loss function. The specific expression is: ; in, for The regularization coefficient of regularization, These are all learnable parameters in the PRSM model.

[0061] Step S6: Based on the aspect category prediction results and the sentiment polarity classification results, integrate and output the prediction results.

[0062] Based on the output results of steps S4 and S5, the first emotional polarity label is calculated. Prediction results of each sentiment polarity-aspect category combination The specific expression is: ; in, Let be the set of points representing the maximum value of the independent variable.

[0063] For three different sentiment polarity labels (positive, neutral, and negative), the above process is performed separately, and the results are integrated to finally output the prediction results of aspect category sentiment analysis. Specifically: For the input sequence to be analyzed, it is first concatenated with the three sentiment polarity labels "positive," "neutral," and "negative," respectively, and treated as three different inputs, which are then fed into the PRSM model. For each sentiment polarity label, the PRSM model first uses the sentiment classification module to determine "whether the sentence contains that sentiment polarity," outputting a binary classification result (0 indicates absence, 1 indicates presence). Then, under the current sentiment polarity condition, the aspect category classification module performs multi-label judgments on all aspect categories, outputting whether each aspect category appears (0 indicates absence, 1 indicates presence). Thus, for each sentiment polarity, a set of prediction results is obtained: "whether the sentence has that sentiment polarity (0 or 1) + whether each aspect category is mentioned under that sentiment polarity (0 or 1)."

[0064] The final "integration" is not simply averaging, voting, or normalizing the outputs from the three rounds, but rather a logical concatenation: for each "sentiment polarity – aspect category" combination, if the sentiment polarity is determined to exist (value 1), and the aspect category is also determined to exist (value 1) under that sentiment polarity, then the sentence is considered to contain this "aspect – sentiment polarity" pair. Collecting all combinations determined to exist (value 1) under the three sentiment polarities constitutes the final aspect category sentiment analysis prediction result. Therefore, the essence of integration is: performing multi-label predictions under the three sentiment polarity conditions separately, and then summarizing and outputting all "aspect category – sentiment polarity" pairs formed by the three predictions in a grid-like manner, rather than performing additional fusion operations on the prediction results of the three sentiment polarity-aspect category combinations.

[0065] Based on the above processing, the final prediction result for the "hamburger" example sentence is: {(food # quality, positive), (service # average, negative)}. This result accurately reflects the user's praise for the food itself and disappointment with the details of the service. This result also aligns with the example in step S4 mentioned above. <s> neutral< / s> The intermediate conclusion that the sequence did not produce a high-probability prediction corroborates each other, jointly proving that the PRSM model can effectively identify the actual sentiment tendencies in the original text, without forcibly outputting non-existent neutral viewpoints.

[0066] To verify the effectiveness of the method in this embodiment, evaluation experiments were conducted using four publicly available datasets from SemEval-2015 and SemEval-2016 (namely Restaurants15, Laptops15, Restaurants16, and Laptops16). These datasets cover a range of restaurant and laptop reviews, with each text annotated with its respective aspect category and sentiment polarity. Table 2 lists the number of sentences in the training and validation sets for each dataset, along with the corresponding number of aspect categories and sentiment polarity (including positive, neutral, and negative) annotations.

[0067] Table 2 Statistical information of the experimental dataset

[0068] All experiments were implemented using the PyTorch deep learning framework. The proposed PRSM model employs the following configuration: a pre-trained RoBERTa-based model is used as the encoder, with the word embedding dimension set to 768. A pre-trained T5-based model is also used as the autoregressive module. In the AdamW optimizer, the learning rate of the aspect classification module is set to 2e. –3 The learning rate for other parameters is set to 2e. –5 Other hyperparameters used in the experiment are shown in Table 3. The precision (P) and recall (R) of the prediction results were calculated through the experiment, and the F1 score was used as the final evaluation metric. To ensure the stability of the experimental results, the average of five independent runs was taken as the final result.

[0069] Table 3 Hyperparameter Settings

[0070] To comprehensively evaluate the performance of the PRSM model, it was trained and evaluated on four datasets, including: (List of datasets would be inserted here) (1) Pipeline-BERT: An ACD-SC pipeline model using BERT as the encoder. (2) Cartesian-BERT: A model using BERT as the sentence encoder and employing the Cartesian product method. (3) Addonedim-BERT: A model using BERT as the sentence encoder and employing the method of adding one dimension. (4) AS-DATJM: A model that uses an attention mechanism in ACD to obtain vector representations of aspect categories, and under the guidance of vectors, SC uses GCN to aggregate sentiment context. (5) Hier-BERT: A model based on BERT with a hierarchical prediction method. (6) Hier-Trans-BERT: A model based on Hier-BERT that uses Transformer to capture the associations between aspect categories and the relationship between these categories and sentiment. (7) Hier-GCN-BERT: A model based on Hier-BERT that uses GCN for relation learning. (8) PBJM: A baseline model that combines manual prompt templates and an attention mechanism.

[0071] As shown in Table 4, the experimental results indicate that Pipeline-BERT performs poorly compared to traditional ACSA models such as Cartesian-BERT, Addonedim-BERT, and AS-DATJM. This highlights the importance of modeling the interaction between aspect category detection and sentiment prediction. Pipeline-BERT employs a pipelined hierarchical classification method, decoupling ACD from SC, making it difficult for the model to effectively capture the correlation between the two subtasks. In contrast, Cartesian-BERT, Addonedim-BERT, and AS-DATJM predict through aspect category-sentiment polarity pairs, thus fully leveraging the interaction between the two. However, Cartesian-BERT only significantly outperforms Pipeline-BERT on the Restaurants15 and Restaurants16 datasets. On the Laptops16 dataset, their performance is similar, and even on the Laptops15 dataset, Cartesian-BERT's F1 score is significantly lower than Pipeline-BERT's. This is likely due to the large number of aspect categories in laptop-type datasets, particularly Laptops15, where most instances may contain multiple aspect categories. Cartesian-BERT uses a Cartesian product method to generate all feasible category-sentiment pair combinations, introducing significant redundant information and substantially increasing computational cost, ultimately leading to a decline in the model's predictive performance. Compared to the previous models, Hier-BERT, Hier-Trans-BERT, Hier-GCN-BERT, and PBJM employ end-to-end modeling methods to more effectively capture the correlation between aspect categories and sentiment polarity.

[0072] Experimental results show that Hier-GCN-BERT significantly outperforms Hier-Trans-BERT in F1 scores on three datasets except Restaurantes15, but performs slightly worse on Restaurantes15. While GCNs can model complex relationships well, the text in the Restaurantes15 dataset is relatively short and the sentence-level categories are relatively simple, potentially leading to an over-reliance on the relationship graph constructed by GCN. However, when the graph structure of the data itself is unclear or the data is noisy, the graph model may be affected, resulting in lower-than-expected performance. PBJM not only enhances task relevance by jointly encoding aspect category prediction and sentiment classification subtasks with BERT, but also introduces manual prompt templates into the input text. By utilizing the masked language modeling mechanism of the pre-trained model, it predicts the aspect category corresponding to the masked markers in the manual prompt template, effectively avoiding interference from redundant information and reducing computational overhead, thereby improving model performance.

[0073] Table 4. Experimental results (%) for the four datasets

[0074] As shown in Table 4, the PRSM model of this embodiment demonstrates the best performance across all datasets, validating its effectiveness and reliability. Compared to PBJM, the PRSM model improves the F1 score by 10.0% on the Laptops15 dataset, 8.9% on Laptops16, 6.7% on Restaurants15, and 5.4% on Restaurants16. This indicates that the PRSM model achieves a greater performance improvement on the Laptops dataset than on the Restaurants dataset. This is because the Laptops dataset has more aspect categories and more complex sentence structures, allowing the PRSM model to perform more detailed and comprehensive learning. In contrast, most instances in the Restaurants dataset contain only one aspect category, making it difficult for the complex model to fully leverage its advantages, thus resulting in a slightly lower performance improvement.

[0075] To evaluate the effectiveness and rationality of each component of the PRSM model, ablation experiments were conducted on four public datasets. PBJM was used as the baseline model for the ablation experiments, and a component was removed or added to it: BERT used in PBJM was replaced with RoBERTa; BiLSTM was introduced into the sentiment classification module of PBJM; and a T5 autoregressive generative model was added to PBJM. The performance of different component combinations was compared to verify the contribution of each improvement to the final model performance. The ablation experiment results are shown in Table 5.

[0076] Table 5 Ablation experiment results (%) on four datasets

[0077] Experimental results show that, compared with the baseline model, introducing RoBERTa, BiLSTM, and T5 autoregressive generative models all improve the F1 scores of the model on the four datasets. Among them, introducing RoBERTa shows the largest improvement, increasing the F1 scores by 6.5%, 5.5%, 4.5%, and 3.4% on the Laptops15, Laptops16, Restaurants15, and Restaurants16 datasets, respectively. Although the improvement in F1 score from introducing the T5 autoregressive generative model is not as significant as that from BiLSTM, its R-value is higher than that of BiLSTM on all four datasets. This phenomenon may be because the T5 autoregressive generative model generates explanatory content related to the original text, enabling the model to better capture implicit information, thereby increasing the number of aspect categories predicted in the final model.

[0078] Compared to introducing RoBERTa, BiLSTM, or T5 autoregressive generative models individually, the F1 scores of models using a combination of RoBERTa-BiLSTM, RoBERTa-T5 autoregressive generative models, or BiLSTM-T5 autoregressive generative models were all improved. This demonstrates that combining multiple components can effectively improve model performance. When using the RoBERTa-BiLSTM-T5 autoregressive generative model combination, which is the complete PRSM model, the F1 scores on the four datasets improved by 10.0%, 8.9%, 6.7%, and 5.4%, respectively, further validating the effectiveness and state-of-the-art of the PRSM model.

[0079] Therefore, this invention employs the aforementioned aspect category sentiment analysis method based on cue prediction and semantic enhancement, achieving efficient and accurate aspect category identification and sentiment polarity judgment, enhancing the ability to understand the deep semantics of short texts, improving adaptability in data-scarce scenarios, and improving the accuracy and robustness of aspect category sentiment analysis.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.< / s> < / mask> < / mask> < / mask> < / mask> < / mask> < / mask> < / mask> < / mask> < / mask> < / mask> < / mask> < / mask> < / mask>

Claims

1. A sentiment analysis method for aspect categories based on cue prediction and semantic enhancement, characterized in that, Includes the following steps: Step S1: Input the original text into the autoregressive module to generate extended text, and then concatenate the extended text with the original text to form enhanced text; Step S2: Construct the input sequence by combining the sentiment polarity labels, enhanced text, and cue templates according to a predefined format; Step S3: The input sequence is encoded by the RoBERTa encoder module to obtain a context vector representation; Step S4: Input the context vector into the aspect category classification module, extract the information vector through the attention function, map it to the probability distribution of the label words, and obtain the aspect category prediction result; Step S5: Input the context vector sequence into the sentiment classification module, obtain a bidirectional LSTM representation through BiLSTM, and obtain the sentiment polarity classification result through feature fusion; Step S6: Based on the aspect category prediction results and the sentiment polarity classification results, integrate and output the prediction results.

2. The aspect category sentiment analysis method based on cue prediction and semantic enhancement according to claim 1, characterized in that, In step S1, the autoregressive module is a T5 autoregressive generative model.

3. The aspect category sentiment analysis method based on cue prediction and semantic enhancement according to claim 1, characterized in that, In step S2, the prompt template uses the fixed sentence structure: This sentence mainly describes... <mask> 。< / mask> 4. The aspect category sentiment analysis method based on cue prediction and semantic enhancement according to claim 1, characterized in that, In step S2, the predefined format is: <s> Emotional polarity tags< / s> Original text + extended text hint template.

5. The aspect category sentiment analysis method based on cue prediction and semantic enhancement according to claim 1, characterized in that, In step S2, three input sequences are constructed, with their sentiment polarity labels corresponding to positive, neutral, and negative, respectively.

6. The aspect category sentiment analysis method based on cue prediction and semantic enhancement according to claim 1, characterized in that, Step S4 specifically includes: The context vector representation is used as the input aspect category classification module, and the full-text vector representation is extracted and... <mask> The labeled vector is used to calculate the association weight between the two through an attention function, and the aspect-related information vector is extracted.< / mask> Connect information vectors with <mask> The label vectors are concatenated and dimensionality reduced to obtain the overall predicted label word vectors;< / mask> The label word vectors are mapped to a pre-created vocabulary through a RoBERTa pre-trained linear layer, which is then converted into a probability distribution of all words in the vocabulary. Based on probability distribution, the probability of the tag words corresponding to each aspect category is statistically analyzed, and the scores corresponding to each tag are calculated and compared to obtain the aspect category prediction results.

7. The aspect category sentiment analysis method based on cue prediction and semantic enhancement according to claim 1, characterized in that, In step S5, feature fusion specifically involves extracting the first label from the context vector representation. <s> The vector representation is then concatenated and fused with the bidirectional LSTM representation to form a fused feature vector.< / s> <s> 8. The aspect category sentiment analysis method based on cue prediction and semantic enhancement according to claim 7, characterized in that, Step S5 specifically includes: The context vector sequence is input into the sentiment classification module, and the last hidden state of the forward LSTM and the first hidden state of the backward LSTM are obtained through BiLSTM respectively. The last hidden state of the forward LSTM and the first hidden state of the backward LSTM are concatenated to obtain the bidirectional LSTM representation. Perform feature fusion: Extract the first label from the context vector representation. <s> The vector representation is then concatenated and fused with the bidirectional LSTM representation to form a fused feature vector.< / s> <s> The fused feature vectors are passed through a fully connected layer and a softmax activation function to output the sentiment polarity classification result.

9. The aspect category sentiment analysis method based on cue prediction and semantic enhancement according to claim 1, characterized in that, The method is executed by a PRSM model that integrates an autoregressive module, a RoBERTa encoder module, an aspect category classification module, and a sentiment classification module.

10. The aspect category sentiment analysis method based on cue prediction and semantic enhancement according to claim 9, characterized in that, The PRSM model is optimized through end-to-end joint training, with a total loss function. Loss function based on aspect category prediction And sentiment classification loss function Together they constitute, and the specific expression is: ； ； ； in, For real labels, For the number of aspect categories, For the first The categories in the first Predicted probabilities on sentiment polarity labels As a label for the true polarity of emotions, For the first The predicted probability of each sentiment polarity label for The regularization coefficient of regularization, These are all learnable parameters in the PRSM model. < / s> < / s>

Images