LLM-based irony speech understanding method

Abstract

The invention discloses an LLM-based irony speech detection and understanding method, and the method comprises the steps: S101, constructing a multi-task irony data set; wherein the multi-task irony data set comprises a plurality of samples of a plurality of tasks, and the tasks comprise irony detection, irony interpretation, irony style conversion and irony fine-grained classification; step S102, based on a low-rank adaptive technology, performing multi-task hybrid fine tuning on the large language model by using the multi-task irony data set to obtain a fine-tuned large language model; and S103, inputting to-be-understood text information into the fine-tuned large language model to obtain an irony speech understanding result. In the method, when the fine-tuned large language model carries out irony speech understanding on the to-be-understood text information, high performance can be kept on multiple tasks, and the model training cost is reduced.

Description

Technical Field

[0001] This invention belongs to the field of satirical speech analysis technology, specifically involving a satirical speech understanding method based on LLM. Background Technology

[0002] The Cihai dictionary defines "satire" as the use of mockery and ridicule to expose ugly or backward social phenomena and absurd behaviors. It often conceals negative emotions beneath a seemingly positive surface (Chaudhari and Chandankhede, 2017). Today, satire is widespread on social media platforms, and its automatic detection is of significant value in various tasks and applications that rely on understanding users' true emotional states, such as customer service, opinion mining, public opinion analysis, and online harassment detection. Existing satire detection models have significant limitations when facing diverse satire understanding tasks, such as difficulty in maintaining high performance across multiple tasks simultaneously. Training a complete task model for each task would significantly increase model training costs. Therefore, there is an urgent need for an LLM-based satire understanding method that can maintain high performance across multiple tasks while reducing model training costs.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0005] This disclosure provides an LLM-based method for understanding satirical remarks, which can maintain high performance across multiple tasks and reduce model training costs.

[0006] Preferably, the LLM-based method for understanding satirical remarks includes: step S101, constructing a multi-task satirical dataset; wherein, the multi-task satirical dataset includes multiple samples from multiple tasks, the tasks of which include satirical detection, satirical interpretation, satirical style transfer, and fine-grained satirical classification; step S102, using low-rank adaptive techniques, fine-tuning the large language model using the multi-task satirical dataset to obtain a fine-tuned large language model; step S103, inputting the text information to be understood into the fine-tuned large language model to obtain the satirical remarks understanding result.

[0007] The beneficial effects of this invention are as follows: By acquiring multiple samples from different tasks and aggregating them into a multi-task satirical dataset, a low-rank adaptive technique is used to fine-tune a large language model across multiple tasks using this dataset, resulting in a fine-tuned model. During fine-tuning, the low-rank adaptive technique freezes the original parameters of the large language model, preserving its original natural language processing capabilities. By combining training with the various task subsets, the fine-tuned model retains its original natural language processing abilities while learning knowledge from different tasks, maintaining high performance across multiple tasks. Thus, when interpreting satirical statements, this fine-tuned large language model can maintain high performance across multiple tasks while reducing training costs.

[0008] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0009] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein: Figure 1 This is a flowchart of an LLM-based method for understanding satirical remarks provided by the present invention; Figure 2 This invention provides a histogram showing the distribution of prompt word length and sample quantity. Figure 3 This is a schematic diagram of a sample provided by the present invention before and after adding the corresponding prompt template; Figure 4 This is a logical diagram of an LLM-based method for understanding satirical remarks provided by the present invention; Figure 5 This is a classification result diagram of all models before the application of the prompt word enhancement and data augmentation strategy provided by this invention; Figure 6 This invention provides a classification result diagram of all models after applying prompt word enhancement and data augmentation strategies. Detailed Implementation

[0010] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0011] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0012] Unless otherwise stated, the term "multiple" means two or more.

[0013] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0014] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0015] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0016] Irony often conveys emotions contrary to the literal meaning and is frequently used as a strategy to circumvent content moderation. Such language can easily exacerbate conflicts and confrontations within online communities. Previous models were typically optimized for specific aspects of irony processing (such as detection or interpretation). With the advent of Large Language Models (LLMs), their pure decoder architecture allows a single model to perform multiple tasks simultaneously. The multi-task irony dataset (SarcasmCN) provided in this disclosure is a comprehensive Chinese dataset supporting multi-task irony understanding, covering five different tasks.

[0017] Large Language Models (LLMs) are deep learning models trained on massive amounts of text. They possess powerful natural language understanding and generation capabilities, enabling them to perform various tasks such as complete text generation, question answering, and translation by learning human language patterns. Furthermore, as an autoregressive model, LLM's ability to predict the next token allows it to unify different tasks into a single text generation framework, significantly improving the model's generalization ability. Low-rank approximation (LoRa) is a parameter compression technique based on matrix factorization. By introducing low-rank matrix factorization, it updates only the decomposed matrix, rather than the entire model parameters. This method significantly reduces the number of training parameters and computational resource requirements while maintaining model performance.

[0018] Combination Figure 1 As shown, this disclosure provides an LLM-based method for understanding satirical remarks, characterized by including: Step S101: Construct a multi-task satire dataset; wherein, the multi-task satire dataset includes multiple samples from multiple tasks, including satire detection, satire interpretation, satire style transfer, and fine-grained satire classification.

[0019] In some embodiments, LLMAPI is used to generate satirical keywords to search for related topics. The original data is deduplicated and anonymized. A human-machine joint annotation framework is constructed, and human evaluation metrics are introduced to assess the scientific validity of the dataset. Data augmentation techniques are employed to expand the original samples; prompt word templates are carefully designed, and standard input is constructed.

[0020] The satire detection task, also known as the satire identification task, is used to identify whether a statement is satirical. The satire explanation task is used to describe the reasons behind satirical statements. The satire style conversion task is used to perform style conversion, transforming satirical statements into non-satirical statements without changing their satirical meaning. Fine-grained satire classification tasks include satire target classification tasks and satire style tasks. The satire target classification task explores the targets of gap statements, such as personal attack satire (satire targeting a specific individual, small group, company, or institution); social criticism satire (satire targeting public policy, social phenomena, systemic problems, or collective behavior); and geopolitical satire (satire targeting international relations, foreign policy, global strategy, ideological conflict, or national sovereignty). The satire style task classifies expression styles, such as explicit satire (direct and explicit satire, the satirical intent can be identified without relying on background knowledge or context); and implicit satire (indirect and subtle satire, requiring context or background knowledge to identify the satirical intent).

[0021] Step S102: Based on low-rank adaptive technology, the large language model is fine-tuned using a multi-task satirical dataset to obtain the fine-tuned large language model.

[0022] That is, the LoRa fine-tuning technique is used to fine-tune the large model.

[0023] Step S103: Input the text information to be understood into the fine-tuned large language model to obtain the understanding result of the satirical remarks.

[0024] This disclosure presents an LLM-based method for understanding satirical remarks. It acquires multiple samples from different tasks and aggregates them into a multi-task satirical dataset. Then, using low-rank adaptive techniques, a large language model is fine-tuned across multiple tasks using this dataset, resulting in a fine-tuned model. During fine-tuning, the low-rank adaptive technique freezes the original parameters of the large language model, preserving its original natural language processing capabilities. By combining training with the datasets from various tasks, the fine-tuned model retains its original natural language processing capabilities while learning knowledge from different tasks, maintaining high performance across multiple tasks. Thus, when understanding satirical remarks in text, this fine-tuned large language model can maintain high performance across multiple tasks while reducing model training costs.

[0025] Research on sarcasm detection in the field of Natural Language Processing (NLP) has been extensively explored, especially in English sarcasm analysis, where resources are particularly abundant. Most English datasets consider contextual information as a key factor in determining sarcastic intent. In recent years, Arabic sarcasm detection and dataset construction have also received increasing attention. However, existing Chinese sarcasm corpora remain extremely scarce. Our survey shows that currently only two Chinese sarcasm corpora contain contextual information. Although current research mainly focuses on sarcasm recognition, several studies have begun to explore the underlying mechanisms of sarcasm. For example, Ghosh et al. analyzed the contextual triggers that produce sarcastic responses; Kumar et al. proposed the new task of "sarcasm interpretation in dialogue" to reveal implicit meanings; Jing et al. introduced the Multimodal Sarcasm Interpretation (MuSE) task to interpret the causes of sarcasm in multimodal social media posts; and Hong and Gauch studied methods to improve stance detection performance using sarcasm detection. A natural idea is whether we can create a multi-task sarcasm dataset to build a relatively comprehensive framework for sarcasm understanding. In this work, we introduce the first multi-task Chinese sarcasm detection dataset and conduct a comparative analysis of existing Chinese sarcasm datasets with our dataset. In recent years, large language models (LLMs) with decoder-only architectures have been increasingly used in satire research due to their superior language understanding capabilities. From the perspective of model input, Yao et al. proposed the SarcasmCue cue framework to improve the zero-shot satire detection performance of LLMs by explicitly extracting cues. Similarly, Qiu et al. improved domain-specific satire detection by incorporating external common sense knowledge into LLMs. Bojić et al. analyzed the reliability differences between LLMs and humans from multiple dimensions, finding that LLMs have lower stability in satire detection. Gole and Miranskyy studied the interpretation of satirical emoji usage by GPT-4o, showing that the model consistently assigned higher satire scores to emojis compared to human evaluations. Zheng et al. further investigated how different LLM architectures and fine-tuned hyperparameters affect satire detection capabilities. However, current research has not fully utilized the general potential of large language models (LLMs)—whether a single model can simultaneously perform satire detection, interpretation generation, and satire-to-literal translation tasks, and whether cross-task associations can enhance the understanding of satire. To address these issues, we established a baseline for LLMs in satirical multitasking by fine-tuning 12 representative open-source models (including variants of Llama 3, Glm 4, and Qwen 2.5) and conducted a systematic capability evaluation across five tasks. Furthermore, we explored the impact of data augmentation on overall model performance and improved model performance, thus establishing a benchmark.Finally, we provide a data augmentation method for the multi-task sarcasm dataset.

[0026] Preferably, a multi-task sarcasm dataset is constructed, including: obtaining multiple Chinese text information with context; wherein, each Chinese text information includes a topic and a comment; performing data desensitization and data deduplication on each Chinese text information, and respectively annotating each Chinese text information after data desensitization and data deduplication based on the annotation requirements of each task, to obtain a first sub-dataset corresponding to each task; wherein, one task corresponds to one first sub-dataset, and the samples in the first sub-dataset include Chinese text information and labels; determining the sub-datasets of each task based on the expert evaluation method and each first sub-dataset; and determining the multi-task sarcasm dataset based on the sub-datasets of each task. Among them, the topic and the comment have a context relationship.

[0027] In this way, by obtaining multiple Chinese text information and performing data desensitization, data deduplication, and annotation, and then using the expert evaluation method to evaluate the quality of each first sub-dataset, the annotation quality of each sample in the multi-task sarcasm dataset can be better guaranteed.

[0028] In this embodiment, the dataset is independently annotated by multiple experts, that is, multiple experts independently annotate each Chinese text information.

[0029] Preferably, the Chinese text information is a topic-comment pair composed of a topic and a comment. The topic and the comment have a context relationship. For example, topic: <The A-share market generally rose in the morning, led by the top four in technology (artificial intelligence, robotics, computing power / big data, cloud computing / software)...>. Comment: <It's normal for it to fall. The short-selling institutions are eagerly waiting>".

[0030] Specifically, we obtained multiple contextual Chinese text messages through methods including crawling posts from two publicly accessible online media platforms, Weibo and Baidu Tieba. Both platforms represent local Chinese users and have active communities around specific topics. To obtain data from areas more likely to contain satirical content, we limited our data crawling to a dozen or so controversial topics (such as freedom and education). For each topic, we created a set of keywords to search for tweets and selected content with high engagement. We then removed duplicates and filtered out posts with fewer than three comments. Ultimately, we extracted 86,126 topic-comment pairs (i.e., Chinese text messages) with comments. During the data cleaning phase (i.e., data anonymization and deduplication), the data was anonymized, and embedded links and privacy-related data such as @mentions were removed. Emojis, which may carry important emotional cues, were retained. Due to the scarcity of satirical content, the existing context-based Chinese satirical dataset was further refined, and nearly 300 additional Chinese text messages were added to supplement the dataset.

[0031] Understandably, English datasets dominate existing context-aware datasets, while Chinese datasets remain scarce. To promote diversity in satire datasets, a Chinese corpus was chosen as the foundation for dataset construction. Compared to existing satire datasets, the SarcasmCN dataset (i.e., the multi-task satire dataset) has several significant differences and advantages. First, while most publicly available datasets are in English and primarily focus on binary satire classification, SarcasmCN is the first large-scale Chinese satire dataset to support multiple fine-grained tasks, including satire cause identification (i.e., satire explanation), expression type annotation (i.e., satire style), target extraction (i.e., satire target classification), and style transfer (i.e., satire style transfer). Second, unlike datasets that only contain isolated sentences without surrounding context, SarcasmCN explicitly includes context, i.e., topic-comment pairs, to better capture pragmatic cues crucial for understanding satire. Third, the data was collected from different Chinese social media platforms (Baidu Tieba and Weibo), achieving broader topic coverage and greater linguistic diversity. Finally, all annotations were manually curated by native speakers to ensure high label quality, which is essential for training and evaluating robust models. These features make SarcasmCN a comprehensive resource that goes beyond traditional satire detection, facilitating in-depth research into the mechanisms and expressions of Chinese satire.

[0032] Specifically, the Chinese text information after data anonymization and deduplication is annotated, including using machine-generated annotations as a baseline, followed by human annotators revising unreasonable sentences. Since entirely manual annotation is difficult and labor-intensive, this combined machine and human annotation method ensures annotation quality while reducing annotation difficulty and improving work efficiency.

[0033] In this embodiment, after manual annotation, the ratio of satirical comments to non-satirical comments is approximately 1:17. This causes the model to "learn" to consistently predict non-satirical content while still achieving high accuracy, without truly understanding the satirical meaning. To address this imbalance, we randomly removed a portion of non-satirical samples, bringing the ratio of satirical to non-satirical samples to 1:1. That is, in the multi-task satirical dataset, for each task's multiple samples, there are both satirical and non-satirical samples, and the ratio of satirical to non-satirical samples is 1:1.

[0034] Preferably, the subsets for each task are determined based on the expert evaluation method and each first subset, including: determining each second subset based on the expert evaluation method and each first subset; using an intermediate truncation strategy to truncate each sample in each second subset to obtain each third subset; wherein the samples in the third subset are the truncated samples; and using each third subset as the subset for each task.

[0035] Because Weibo posts vary greatly in length, model fine-tuning may become unstable, and the key parts of the tweet content are usually located at the beginning or end. Therefore, to reduce information loss, a truncation strategy is used to truncate topics and comments. This ensures that the processed topic-comment pairs retain core semantic information while balancing the fairness of processing samples of different lengths. Consequently, when fine-tuning a large language model using a multi-task satire dataset, noise interference can be reduced and the model's ability to capture key information can be improved.

[0036] Preferably, a truncation strategy (i.e., "preserve both ends, cut the middle") is used to truncate each sample within each first subset of the dataset. This includes: determining the length of the Chinese text information for each sample within each first subset of the dataset; for Chinese text information longer than a preset length, starting from the middle position of the Chinese text information, counting is performed towards the beginning and end of the Chinese text information, with the number of characters retained being close to or equal to the preset length. The portion exceeding the preset length, i.e., the text extending beyond the counting range from the middle position, is replaced with an ellipsis (...).

[0037] For example, suppose a Weibo post reads, "Today I went to an amazing restaurant with my friends! The ambiance was fantastic, the service was excellent, and the food was absolutely delicious—we raved about every single dish. We also took lots of great photos and plan to go again!" The total length is 90 characters. If the default length is 50 characters: First, find the middle position, approximately the 45th character. Starting from the 45th character, count characters towards the beginning and end, keeping enough characters to ensure the final text length is 50 characters. The processed text might then become, "Today I went to an amazing restaurant with my friends! The ambiance was fantastic… I plan to go again!", with ellipses used to represent the omitted parts.

[0038] In some embodiments, combined with Figure 2 As shown, Figure 2 A histogram showing the distribution of prompt word length versus sample size is provided. Figure a shows the histogram of prompt word length versus sample size distribution before applying the truncation strategy. Figure b shows the histogram of prompt word length versus sample size distribution after applying the truncation strategy.

[0039] Preferably, based on expert evaluation and each first subset of data, each second subset of data is determined, including: Step S201: According to the task type corresponding to each first subset, perform stratified sampling on each first subset to obtain a sampled dataset.

[0040] Specifically, two rounds of stratified sampling can be performed. In each round, the proportion of samples drawn from each first subset of the dataset differs to reduce the impact of annotator bias on data annotation and to minimize the impact of data distribution on annotation consistency. Specifically, in each sampling process, a predetermined number of instances for irony detection and a predetermined number of instances for fine-grained irony classification are extracted to ensure that the annotation reliability reaches a 95% confidence level. For example, the predetermined number is approximately 130.

[0041] Step S202: The Cohen's Kappa coefficient is used to evaluate the annotation quality of the sampled dataset. The Kappa coefficient is used as the evaluation result, and it is determined whether the evaluation result meets the preset evaluation requirements. The evaluation formula for the Cohen's Kappa coefficient is as follows: in, p is the Kappa coefficient. o For observed consistency, p e For expected consistency, N is the total number of samples, and k is the total number of categories. For samples where two evaluators (evaluator A and evaluator B) both give the same label in class i, For evaluator A, the total number of samples labeled as class i is... The total number of samples labeled as class i by evaluator B.

[0042] In this embodiment, three experts with different cultural backgrounds were selected, and they were systematically introduced to the annotation guidelines and task requirements. To minimize the impact of data distribution on annotation consistency, we performed two rounds of stratified sampling on the dataset. Each sample was evenly distributed among the three annotators to mitigate the influence of individual subjectivity. Finally, we averaged the scores from all rounds to obtain the final result, which serves as the evaluation result.

[0043] In step S203, if the evaluation results meet the preset evaluation requirements, each first subset of data will be used as a second subset of data. If the evaluation results do not meet the preset evaluation requirements, each Chinese text information will be re-annotated to obtain a new first subset of data corresponding to each task, and the process will return to step S201 until each second subset of data is obtained.

[0044] Because the subjective bias of annotators (i.e., human annotators) can negatively impact sample labeling, stratified sampling is performed on each first subset of the dataset. Expert evaluation is then used to check the labeling quality of the sampled datasets to determine if they meet the preset evaluation requirements. If the requirements are met, the labeling quality of the samples within each first subset is good and suitable for model training. If the requirements are not met, the labeling quality is poor and unsuitable for model training. In this case, re-labeling and expert evaluation are conducted to obtain the initial dataset. This ensures the quality of each first subset, thereby guaranteeing the quality of the multi-task satire dataset and resulting in a better-performing model during training.

[0045] In this embodiment, the preset evaluation requirement indicates that the Kappa coefficient is greater than a preset threshold.

[0046] Preferably, each Chinese text information is re-annotated to obtain a new first subset of data for each task, including: analyzing the reasons why the evaluation results do not meet the preset evaluation requirements; selecting pending Chinese text information from each Chinese text information based on the reasons, and re-annotating each pending Chinese text information; and obtaining a new first subset of data for each task based on the re-annotated pending Chinese text information and the un-re-annotated Chinese text information.

[0047] In this way, by analyzing the reasons why the evaluation results do not meet the preset evaluation requirements, and based on the reasons, we can select the pending Chinese text information from each Chinese text information, and then annotate each pending Chinese text information, so as to improve the annotation quality and reduce the workload of re-annotation.

[0048] Preferably, based on the cause, pending Chinese text information is selected from various Chinese text information, and each pending Chinese text information is re-annotated, including: if the cause is a personnel revision error (i.e., due to poor quality of individual personnel revision, the evaluation result does not meet the preset evaluation requirements), then the personnel pointed to by the personnel annotation error is taken as the target personnel, and the Chinese text information revised by the target personnel is taken as pending Chinese text information, and each pending Chinese text information is re-annotated. If the cause is an annotation rule error (i.e. due to incorrect annotation rules, the quality of machine annotation and personnel revision is poor, resulting in the evaluation result not meeting the preset evaluation requirements), then the incorrect annotation rule is revised, the Chinese text information corresponding to the incorrect annotation rule is taken as pending Chinese text information, and each pending Chinese text information is re-annotated according to the revised annotation rule.

[0049] Preferably, a truncation strategy is used to truncate each sample in each second subset to obtain each third subset, including: truncating each sample in each second subset using a truncation strategy to obtain each fourth subset; performing a lookup operation in a preset data table based on the task type corresponding to each fourth subset to find the prompt template corresponding to each task type; wherein the preset data table stores the correspondence between task types and prompt templates; and adding the corresponding prompt template to the samples in each fourth subset to obtain the third subset.

[0050] In this way, the truncation strategy can minimize the loss of textual information, and the corresponding cue templates for each fourth subset can be quickly determined by looking up a table. Then, the corresponding cue templates are added to the samples within each fourth subset, resulting in a third dataset with added cue templates. This improves the quality of the training samples. During model training, samples with cue templates can guide the model output, effectively improving efficiency and reducing errors.

[0051] Such as combination Figure 3 As shown in the illustration, this disclosure provides a sample before and after adding the corresponding prompt template. The prompt template for this sample is the prompt template for the sarcasm detection task.

[0052] Preferably, a truncation strategy is used to truncate each sample in each second subset to obtain each third subset, including: using a truncation strategy to truncate each sample in each second subset to obtain each fifth subset; performing synonym replacement on the topics and / or comments of the Chinese text information of the samples in each fifth subset, and using the set of Chinese text information before and after replacement as the third subset to obtain each third subset.

[0053] In this way, by performing synonym replacement on the topics and / or comments of the Chinese text information of the samples in each fifth subset, new training samples are generated through simple word-level operations, thereby improving the generalization ability of the model and expanding the amount of data.

[0054] In some embodiments, for both sarcasm detection and sarcasm style tasks (explicit / implicit sarcasm classification tasks), synonym replacement is performed on both the topic and the comment, thereby doubling the dataset size. For sarcasm target classification tasks, to avoid altering the target intent by modifying the comment content, only the topic is replaced with synonyms.

[0055] Specifically, synonym replacement, for example, assuming the input sentence is S=[w1, w2, ..., w n Randomly select k words w i Replace it with the corresponding synonym syn(w) i ): S′=[w1,...,syn(w i ..., w n In this way, a new dataset can be obtained through synonym substitution. Assume the original dataset D contains n sentences, each sentence S... i The above operations will enhance the dataset into multiple new sentences Si′. The newly generated dataset D′ is represented as follows: D′=⋃{Si′, S i syn}

[0056] Preferably, the multi-task satirical dataset is determined based on the subsets of each task, including: obtaining the proportion of each task's subset within the multi-task satirical dataset; and selecting the number of samples corresponding to the proportion from each task's subset as the multi-task satirical dataset.

[0057] In this way, by obtaining the proportion of each task, and using this to determine the number of samples for each task in the multi-task satire dataset, a reasonable proportion can be used to ensure that the model learns the features of different tasks in a more balanced way during training, thereby improving the model's performance.

[0058] Preferably, obtaining the proportion of each task's subset within the multi-task satirical dataset includes: each task's subset having the same proportion within the multi-task satirical dataset.

[0059] By ensuring a consistent proportion, the number of samples for each task in the multi-task satire dataset is kept uniform. This allows the model training to take into account the knowledge features of each task, enabling the fine-tuned large language model to perform well on all satire comprehension tasks.

[0060] Preferably, obtaining the proportion of each task's subset within the multi-task satirical dataset includes: using a Bayesian optimization algorithm to determine the proportion of each task's subset within the multi-task satirical dataset.

[0061] In this way, by using the Bayesian optimization algorithm to find the optimal proportion of each task's subset within the multi-task satire dataset, the fine-tuned large language model can achieve good performance on each satire comprehension task.

[0062] Preferably, a Bayesian optimization algorithm is used to determine the proportion of each task's subset within the multi-task satirical dataset, including: Step S301: Sample data from the preset proportion space corresponding to the subsets of each task to determine at least three proportion groups. Each proportion group includes the proportion of the subsets of each task; and the sum of all proportions within a proportion group is 1.

[0063] Specifically, this can be achieved through Latin hypercube sampling (LHS) or uniform random sampling.

[0064] In this embodiment, in order to ensure that the fine-tuned large language model performs well on various irony comprehension tasks, the preset proportion space should be kept as balanced as possible. Therefore, the preset proportion spaces are [0.15, 0.25].

[0065] Step S302: Determine the initial multi-task satire dataset corresponding to each percentage group, and determine the accuracy corresponding to each percentage group based on each initial multi-task satire dataset. One percentage group corresponds to one initial multi-task satire dataset.

[0066] Specifically, the accuracy corresponding to each proportion group is determined based on each initial multi-task satirical dataset. This includes: using low-rank adaptive techniques, fine-tuning the large language model using each initial multi-task satirical dataset to obtain each initial fine-tuned model, and determining the accuracy of each initial fine-tuned model using a validation set. Each initial multi-task satirical dataset corresponds to one initial fine-tuned model. The validation set for each initial fine-tuned model is the same. The validation set includes multiple samples corresponding to each task, and the proportion of each task within the validation set is consistent, all being 0.2.

[0067] Step S303: Determine the optimal percentage group based on each percentage group and the corresponding accuracy rate, and construct a Gaussian process model.

[0068] Gaussian process models are used to determine the mapping relationship between percentage groups and accuracy. The construction of Gaussian process models is a fairly standard existing technique, and will not be elaborated on here.

[0069] Step S304: Update the Gaussian process model based on the optimal proportion group to obtain a new Gaussian process model, and determine a new optimal proportion group based on the new Gaussian process model. Iterate through step S304 until the termination condition is met, and determine the proportion of each task's subset in the multi-task dataset based on the optimal proportion group corresponding to the termination condition.

[0070] In some embodiments, the termination condition is reaching the maximum number of iterations, or in N consecutive rounds (N is 3 or 5), the difference between the accuracy of the best proportion group in the current round and the accuracy of the best proportion group in the previous round is less than a preset difference threshold.

[0071] Specifically, updating the Gaussian process model based on the optimal proportion group to obtain a new Gaussian process model includes: performing Latin hypercube sampling based on the optimal proportion group to obtain at least three candidate proportion groups; calculating the expected improvement value (EI) of each candidate proportion group based on the Gaussian process model and the expectation function; and selecting the candidate proportion group with the largest expected improvement value as the new optimal proportion group. The Gaussian process model is then updated based on the new optimal proportion group to obtain a new Gaussian process model.

[0072] In this way, the candidate proportion groups obtained through Latin hypercube sampling can cover the potential region of excellence relatively evenly in the parameter space. Furthermore, calculating the expected improvement value of each candidate proportion group based on the expectation function can optimize search efficiency and improve the efficiency of determining the optimal proportion group.

[0073] Specifically, the expected improvement (EI) for each candidate proportion group is calculated based on the Gaussian process model and the expectation function. The candidate proportion group with the largest expected improvement is selected as the new optimal proportion group. This includes obtaining the best accuracy and determining the predicted mean and standard deviation for each candidate proportion group based on the Gaussian process model. For each candidate proportion group, the expected improvement is calculated based on the corresponding predicted mean and standard deviation using the following formula: in, Characterizes the expected improvement value of the i-th candidate proportion group. The predicted mean of the i-th candidate proportion group is represented by the output of the Gaussian process model. The standard deviation of the i-th candidate proportion group in the output of the Gaussian process model. Characterizing the best accuracy, and The standard normal distribution function is used to characterize the target proportion. The candidate proportion group with the largest expected improvement is taken as the new optimal proportion group. Here, optimal accuracy represents the highest accuracy achieved across all historical rounds.

[0074] In this way, by using the expectation improvement function to quantify the improvement potential of the candidate proportion group, the most valuable candidate proportion group can be quickly determined, which greatly improves the efficiency of finding the best proportion group in a high-dimensional parameter space.

[0075] Specifically, the Gaussian process model is updated based on the new optimal proportion set to obtain a new Gaussian process model. This includes: determining a new initial multi-task satire dataset based on the new optimal proportion set, and determining the accuracy corresponding to the new optimal proportion set based on the new initial multi-task satire dataset. The Gaussian process model is then updated based on the new optimal proportion set and the corresponding accuracy to obtain a new Gaussian process model.

[0076] In this way, by continuously updating the Gaussian process model, the accuracy of the Gaussian process model can be improved, which is beneficial for determining and finding the optimal proportion group.

[0077] Preferably, based on low-rank adaptive techniques, a large language model is fine-tuned using a multi-task satirical dataset to obtain a fine-tuned large language model. This includes: when fine-tuning the large language model using a multi-task satirical dataset, a low-rank decomposition method is used to constrain the parameter updates of the training weight matrix of the large language model. The calculation formula is as follows: in, The training weight matrix representing the large language model. Characterization pairs Incremental updates, For the first matrix, Let r be the second matrix, and r represent the intrinsic rank; iterative training continues until the convergence condition is met, resulting in a finely tuned large language model.

[0078] In this way, by using low-rank adaptive techniques to fine-tune the large language model, the original weight matrix of the large language model is frozen during iterative training, and only the parameters of the first and second matrices are updated, thus enabling the fine-tuning of the large language model to be completed relatively quickly.

[0079] Specifically, due to the significant cross-task correlation between the various satire comprehension tasks, this embodiment employs a shared parameter encoder (i.e., setting only one LoRA module (including a first low-rank matrix and a second low-rank matrix)) to handle five similar tasks. This shared parameter encoder is implemented through a LoRA adapter, where the rank r = 8. During training, W0 is frozen and does not receive gradient updates. In this embodiment, the main approach is to utilize a multi-task satire dataset with a consistent sample size ratio for each task to perform low-rank adaptive fine-tuning of the large language model (for higher efficiency). The hyperparameters of the fine-tuned large language model are shown in Table 1, as follows: Table 1 Combination Figure 4As shown, this disclosure provides a logical diagram of an LLM-based method for understanding satirical remarks. In this embodiment, the number and content of samples are increased and enriched through cue word engineering (adding cue templates to samples) and data augmentation (replacing topics and / or comments in Chinese text information with synonyms). The obtained multi-task satirical dataset is then used to perform supervised fine-tuning of the large language model, resulting in higher performance in style transfer (satirical style transfer task), causal explanation (satirical explanation), satirical fine-classification (satirical style task and satirical target classification task), and satirical detection task. In the construction of the multi-task satirical dataset, raw corpora are collected using rich internet corpora, data is cleaned and filtered using rule-based methods, an SFT multi-task hybrid fine-tuning dataset is created, and the Cohen-Kappa human evaluation method is introduced to construct scientifically reliable data; ChineseEDA is used to augment the original data. For the large language model fine-tuning, a LoRa efficient fine-tuning method is adopted, simultaneously fine-tuning multiple tasks, and using rogue, bert_score, blue, p, r, and f as indicators to evaluate the model's overall performance. Compared to traditional single-task fine-tuning, this method can complete multiple tasks simultaneously using only one model, obtaining results for understanding sarcastic remarks. The parameter sharing mechanism for training weights ensures high performance while handling multiple tasks and reduces model training costs.

[0080] Combination Figure 5 and Figure 6 As shown, Figure 5 The classification results of all models before applying cue word enhancement and data augmentation strategies are shown. Figure 6This paper presents the classification results of all models after applying cue word enhancement and data augmentation strategies. Cue enhancement represents adding cue templates, while data augmentation represents synonym replacement. In this embodiment, 12 open-source large-scale language models with parameter sizes ranging from 0.5B to 9B were selected, covering architectures such as Llama, GLM, and Qwen. These models were used as baseline models for the experiment. To ensure consistency and fully utilize the potential of different models, we fine-tuned all models using LoRA on our multi-task satire dataset (to improve experimental efficiency, the sample size ratio for each task in the multi-task satire dataset in this embodiment was consistent). LoRA provides an efficient fine-tuning method for large language models. Furthermore, a unified instruction format was adopted on all models. The dataset was divided into training, validation, and test sets in a 7:1.5:1.5 ratio, and classification performance was evaluated using precision (P), recall (R), and F1 score (F1), while sentence generation ability was evaluated using Rouge-1 (R1), BLEU-4, and SentBERT similarity scores. The optimizer used was AdamW. To maintain consistency, we used a fixed number of training steps, given the diversity in model size and number of parameters. The number of steps and other hyperparameters were determined based on the model's performance on the validation set. All experiments were conducted on four GeForce RTX 4090D GPUs.

[0081] We analyzed the classification predictions of the baseline model on the test set. First, we found that in the task of classifying sarcasm, both small and large models are prone to misclassifying implicit sarcasm (e.g., "The more you fight, the richer you get! You're winning so badly!" "Whose subordinate is this? He's so talented, come join our company!"). These examples clearly require contextual understanding, while humans can easily recognize them. We hypothesize that the model is inefficient at extracting topical information; instead, the topical information may act more as noise, thus affecting performance.

[0082] To address this, we reconstructed fine-grained classification prompts by removing topic information, encouraging the model to focus more on the comment itself. Furthermore, we attempted to inject brief inference processes into the model input without significantly increasing the input length. Experiments on the test dataset show that introducing inference slightly improves the classification performance of smaller models, but this effect becomes negligible as the model size increases. Therefore, we retained the improved prompts in the classification task. In the sentence generation task, the model with larger parameter sizes consistently outperformed smaller models in Rouge-1, BLEU-4, and Sent-BERT scores. We hypothesize that larger models are able to capture finer-grained semantic differences and better reveal underlying sarcastic intentions. In scenarios with limited sample resources, data augmentation is an effective way to address class imbalance in the dataset. To enhance the diversity of the corpus, we used ChineseTextEDA—a third-party Python toolkit that supports common augmentation techniques such as synonym replacement. For example, synonym replacement, assuming the input sentence is S=[w1, w2, ..., w...] n Randomly select k words w i Replace it with the corresponding synonym syn(w) i ): S′=[w1,...,syn(w i ..., w n In this way, a new dataset can be obtained through synonym substitution. Assume the original dataset D contains n sentences, each sentence S... i The above operations will enhance the dataset into multiple new sentences Si′. The newly generated dataset D′ is represented as follows: D′=⋃{Si′, S i syn}

[0083] Specifically, for sarcasm detection and explicit / implicit sarcasm style tasks, we perform synonym replacement on both the topic and the comment, thus doubling the dataset size. However, for the sarcasm target identification task, to avoid altering the target intent by modifying the comment content, we only perform synonym replacement on the topic.

[0084] Figure 6The classification results of all models after applying cue augmentation and data augmentation strategies are presented. After applying cue augmentation and data augmentation, the classification performance of all models generally improved without negatively impacting their ability to perform ironic reasoning and interpretation tasks. Notably, in explicit / implicit irony recognition tasks, the models still made significant progress even after removing topic-related information, consistent with our previous hypothesis. It is worth mentioning that the models did not show substantial changes in sentence generation ability, mainly due to the probabilistic nature of large language models (LLMs)—which rely on statistically significant next-token predictions. Perturbations tend to have a more significant impact on single-token predictions than multi-token outputs. Performance on all evaluation metrics improved significantly with increasing model size. Particularly in the irony target recognition task, the F1 score difference between the best and worst performing models reached 21.67%. However, in the irony detection task, the performance gap between small and large models was relatively small. Given that small models have a smaller parameter size and are easier to deploy, they still have practical value. Furthermore, all models significantly outperformed fine-grained irony classification tasks in irony detection tasks. From the perspective of sentence feature extraction, this indicates that the model's ability to capture subtle emotional cues remains limited.

[0085] In addition, such as Figure 6 As shown, we selected the relatively weaker Qwen2.5-0.5B model and the relatively better GLM4-9B model for comparative analysis. Although both models performed well in the irony detection task, their performance declined to some extent when dealing with more specific ironic expressions and targets. In terms of ironic expressions, both models were better at detecting explicit irony, indicating the presence of more identifiable features in the sentences. However, implicit irony, as a more subtle form of expression, was more difficult to detect because its meaning changed with context. Introducing common-sense knowledge or experience might improve the model's predictive ability for this type of irony. In the classification tasks of geopolitical irony and social critique irony, the performance gap between small and large models was more pronounced, highlighting the significant challenge faced by small models in handling complex irony types that require deeper semantic understanding and knowledge reasoning. The models faced significant challenges in accurately identifying ironic targets. The poor performance in fine-grained classification tasks was also reflected in the results of the irony interpretation task. GLM4's SentBERT score is only 65.62, suggesting that although the model may appear to know what irony is on the surface, it may not truly understand the real meaning of irony.

[0086] As can be seen, understanding the true meaning of satirical remarks is becoming increasingly important due to their widespread prevalence. This disclosure focuses on satirical interpretation, introducing the multi-task Chinese satirical dataset SarcasmCN, which includes five tasks to evaluate models from different dimensions. We evaluated 12 large language models on these tasks and improved their performance through cue engineering and data augmentation. Experiments show that these methods are effective, especially in resource-constrained scenarios. Error analysis reveals a significant gap between model understanding and human annotation, particularly for context-dependent satire and the identification of the satirical target. Translating satirical comments into straightforward expressions also remains a challenge. These findings highlight the necessity of incorporating common-sense knowledge to enhance the model's semantic understanding capabilities.

[0087] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. A sarcasm understanding method based on LLM, characterized in that, The method comprises the following steps: Step S101, constructing a multi-task sarcasm data set; wherein the multi-task sarcasm data set comprises a plurality of samples of a plurality of tasks, and the tasks comprise sarcasm detection, sarcasm explanation, sarcasm style conversion and sarcasm fine-grained classification; Step S102, based on a low-rank adaptive technology, using the multi-task sarcasm data set to perform multi-task mixed fine-tuning on a large language model to obtain a fine-tuned large language model; Step S103, inputting to-be-understood text information into the fine-tuned large language model to obtain sarcasm speech understanding results.

2. The method of claim 1, wherein, The method for constructing the multi-task sarcasm data set comprises: Obtaining a plurality of Chinese text information with context; wherein each Chinese text information comprises a topic and a comment; Performing data desensitization and data deduplication on each Chinese text information, and respectively labeling each Chinese text information after data desensitization and data deduplication based on labeling requirements of each task to obtain a first sub-data set corresponding to each task; wherein one task corresponds to one first sub-data set, and the samples in the first sub-data set comprise Chinese text information and labels; Determining a sub-data set of each task based on the expert evaluation method and each first sub-data set; Determining the multi-task sarcasm data set based on the sub-data set of each task.

3. The method of claim 2, wherein, The method for determining the sub-data set of each task based on the expert evaluation method and each first sub-data set comprises: Determining each second sub-data set based on the expert evaluation method and each first sub-data set; Performing truncation processing on each sample in each second sub-data set by using an intermediate truncation strategy to obtain each third sub-data set; wherein the samples in the third sub-data set are the samples after the truncation processing; Taking each third sub-data set as the sub-data set of each task.

4. The method of claim 3, wherein, The method for determining each second sub-data set based on the expert evaluation method and each first sub-data set comprises: Step S201, stratified sampling each first sub-data set according to the task type corresponding to each first sub-data set to obtain a sampling data set; Step S202, evaluating the labeling quality of the sampling data set by using a Cohen's Kappa coefficient, taking the Kappa coefficient as an evaluation result, and judging whether the evaluation result meets a preset evaluation requirement; wherein the evaluation formula of the Cohen's Kappa coefficient is as follows: wherein, is the Kappa coefficient, p o is the observed agreement, p e is the expected agreement, N is the total number of samples, k is the total number of classes, is the number of samples for which both raters gave the same label for class i, is the total number of samples for which rater A labeled class i, is the total number of samples for which rater B labeled class i; Step S203, if the evaluation result meets the preset evaluation requirement, taking each first sub-data set as each second sub-data set, and if the evaluation result does not meet the preset evaluation requirement, re-labeling each Chinese text information to obtain a new first sub-data set corresponding to each task, and returning to execute step S201 until each second sub-data set is obtained.

5. The method of claim 3, wherein, The method for performing truncation processing on each sample in each second sub-data set by using the intermediate truncation strategy to obtain each third sub-data set comprises: Performing truncation processing on each sample in each second sub-data set by using an intermediate truncation strategy to obtain each fourth sub-data set; The look-up table operation is respectively performed in the preset data table based on the task types corresponding to the fourth sub-data sets, and the prompt templates corresponding to the task types are found out; wherein, the preset data table stores the corresponding relationship between the task types and the prompt templates; The corresponding prompt templates are added to the samples in each fourth sub-data set respectively, and a third sub-data set is obtained.

6. The method of claim 3, wherein, The intermediate truncation strategy is used to perform truncation processing on each sample in each second sub-data set, and each third sub-data set is obtained, including: The intermediate truncation strategy is used to perform truncation processing on each sample in each second sub-data set, and each fifth sub-data set is obtained; Synonyms are replaced for the topics and / or comments of the Chinese text information of the samples in each fifth sub-data set, and the set of the Chinese text information before and after the replacement is taken as a third sub-data set, and each third sub-data set is obtained.

7. The method of claim 1, wherein, The multi-task satire data set is determined based on the sub-data sets of each task, including: The proportion of each task sub-data set in the multi-task satire data set is obtained; The number of samples corresponding to the proportion is selected from each task sub-data set as the multi-task satire data set.

8. The method of claim 7, wherein, The proportion of each task sub-data set in the multi-task satire data set is obtained, including: The proportions of the sub-data sets of each task in the multi-task satire data set are the same.

9. The method of claim 7, wherein, The proportion of each task sub-data set in the multi-task satire data set is obtained, including: The proportion of each task sub-data set in the multi-task satire data set is determined by using a Bayesian optimization algorithm.

10. The method of claim 1, wherein, The multi-task mixed fine-tuning of the large language model is performed by using the multi-task satire data set based on the low-rank adaptive technology, and the fine-tuned large language model is obtained, including: When the multi-task mixed fine-tuning of the large language model is performed by using the multi-task satire data set, the parameter update of the weight matrix to be trained of the large language model is constrained by using a low-rank decomposition method, and the calculation formula is as follows: wherein, a matrix of to-be-trained weights of a large language model, characterizing an incremental update to a matrix of to-be-trained weights of a large language model, is a first matrix, is a second matrix, and r characterizes an intrinsic rank; Iterative training is performed until the convergence condition is met, and the fine-tuned large language model is obtained.

Images