An emotion analysis method based on emotion thought chain reasoning enhancement and knowledge distillation

Abstract

The application discloses a kind of based on emotional thinking chain reasoning enhancement and knowledge distillation emotional analysis method, comprising: the sample of text, image is collected, and real emotional label is marked;Teacher model is used to reason, according to the reasoning result, first real label distillation dataset, second real label distillation dataset is successively constructed, and real label distillation dataset is formed by merging;Based on teacher model, build tutor model and total loss function, the multi-task learning training of tutor model is carried out, based on the reasoning result of tutor model, soft label distillation dataset is constructed;Student model and total loss function are constructed, based on soft label distillation dataset, real label distillation dataset, the fine-tuning and training of student model are carried out;The student model after training is deployed in target application environment, and the sentiment analysis to text and image content is completed.The application can realize the identification of the sentiment polarity of specific aspect words in text, complete the joint understanding and fine-grained reasoning of text and image.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of natural language processing and computer vision, specifically to a sentiment analysis method based on sentiment-based reasoning enhancement and knowledge distillation. Background Technology

[0002] Multimodal aspect-level sentiment analysis (MPA) is an important research direction at the intersection of natural language processing and computer vision. Its goal is to automatically identify the sentiment polarity (e.g., positive, negative, neutral) of specific aspect terms from joint inputs containing text and images. Unlike traditional document-level or sentence-level sentiment analysis, MPA requires models with fine-grained semantic understanding and cross-modal reasoning capabilities, enabling them to distinguish the sentiment tendencies corresponding to different aspect terms within the same sentence. With the explosive growth of image-rich content on social media, MPA has broad application prospects in areas such as opinion mining and product evaluation.

[0003] Currently, mainstream technical methods can be mainly categorized as follows:

[0004] (1) Cross-modal fusion methods based on attention mechanisms achieve interaction modeling of text and image features within and between modalities by designing complex attention networks. For example, the ESAFN model proposes an entity-sensitive attention and fusion network to capture key information interactions between modalities. However, these methods often focus on feature alignment and weighting, lacking explicit modeling of the logic behind sentiment judgment, resulting in poor interpretability of the model.

[0005] (2) Methods that utilize visual emotion cues to extract more explicit emotional signals from images. For example, the FITE model introduces facial emotions in images as visual emotion cues and converts them into emotional text descriptions to aid decision-making. The effectiveness of this type of method is highly dependent on the extraction quality of specific visual cues such as facial emotions, and its performance improvement is limited when faced with images that do not contain faces or have relatively subtle emotional expressions.

[0006] (3) Methods that enhance fine-grained interaction modeling: In order to more accurately associate images with aspect words, models such as MPFIT and CoolNet enhance the deep interaction between aspect words and image regions. By constructing cross-modal fine-grained alignment networks or introducing syntactic knowledge, the fusion efficiency is improved. Although these methods have made progress in performance, their model structures are becoming increasingly complex, and the reasoning process is still similar to a "black box," making it difficult to provide human-understandable judgment criteria.

[0007] (4) Methods based on visual language pre-training frameworks: With the development of large pre-trained models, models such as VLP-MABSA have designed unified encoding and decoding architectures to adapt to various multimodal pre-training tasks, thereby improving fine-grained alignment and recognition capabilities. Although these methods perform well in terms of accuracy and other metrics, their large number of parameters and high computational cost severely limit their deployment in resource-constrained scenarios (such as mobile devices and edge computing). At the same time, although large models have stronger reasoning potential, their complex decision-making processes also lack interpretability.

[0008] In summary, existing technologies face the following common problems when dealing with multimodal aspect-level sentiment analysis tasks: First, the model decision-making process lacks interpretability. Most methods only output the final sentiment polarity and cannot provide the reasoning path to reach that conclusion, which is difficult to meet application requirements in scenarios requiring high-risk decision-making or result traceability. Second, lightweight models have insufficient reasoning capabilities. Although methods based on pre-trained large models have superior performance, they are difficult to deploy at edge computing environments. Lightweight models, due to parameter size limitations, struggle to learn complex cross-modal sentiment reasoning logic. Third, soft labels are missing during the knowledge distillation process. Directly using the hard labels (i.e., the final classification results) of the large model (teacher model) to distill the small model (student model) results in the loss of rich probabilistic information output by the large model. Furthermore, large models are often unable to be deployed locally to generate soft labels due to excessively large parameters, interface limitations, or commercial reasons, leading to low knowledge transfer efficiency.

[0009] Therefore, how to improve the sentiment reasoning ability and interpretability of the decision-making process while ensuring the model's lightweight nature, and how to effectively utilize the knowledge of large models to guide the training of small models, are the technical problems that urgently need to be solved in the field of multimodal aspect-level sentiment analysis. Summary of the Invention

[0010] The purpose of this invention is to provide a sentiment analysis method based on sentiment reasoning chain enhancement and knowledge distillation. This method can solve the technical problems in the existing technology, such as the lack of interpretability of multimodal sentiment analysis models, the difficulty of small models to have sentiment reasoning ability, and the limited knowledge transfer effect caused by the lack of soft labels.

[0011] To achieve the above functions, this invention designs a sentiment analysis method based on sentiment-based reasoning enhancement and knowledge distillation, which executes the following steps S1-S4 to complete sentiment analysis of text and image content:

[0012] Step S1: Collect multiple samples containing text and related images, and label them with corresponding real sentiment tags; based on the real sentiment tags of the samples, use the teacher model to perform inference, generate sentiment thought chain inference content and predicted sentiment tags, and include the samples corresponding to the correct predicted sentiment tags into the first real label distillation dataset; for the samples corresponding to the incorrect predicted sentiment tags, regenerate the predicted sentiment tags based on the real sentiment tags, and include the samples corresponding to the regenerated correct predicted sentiment tags into the second real label distillation dataset. Merge the first real label distillation dataset and the second real label distillation dataset to form the real label distillation dataset.

[0013] Step S2: Based on the teacher model, construct the teaching assistant model and the corresponding total loss function, train the teaching assistant model through multi-task learning, use the trained teaching assistant model to infer the samples in the real label distillation dataset, and construct the soft label distillation dataset based on the inference results.

[0014] Step S3: Construct a student model and the corresponding soft-label loss function and real-label learning loss function. Fine-tune the student model based on the soft-label distillation dataset and the real-label distillation dataset, respectively. Further construct the total loss function of the student model and train the student model based on the real-label distillation dataset.

[0015] Step S4: Deploy the trained student model in the target application environment to complete sentiment analysis of text and image content.

[0016] As a preferred technical solution of the present invention, the specific steps of step S1 are as follows:

[0017] Step S1.1: Extract the i-th sample text Related images and aspect word sequence The aspect word sequence The text is The subsequence for the i-th sample Pre-label the corresponding real emotional tags The categories include positive, negative, or neutral; based on the sample. and the corresponding text Related images Aspect word sequence Authentic emotional tags ,by Data sets are constructed in a formal manner;

[0018] Step S1.2: Based on samples True emotional tags and first prompt word template Using a teacher model The process involves reasoning to generate an emotional thought chain and predicting emotional tags, wherein the first prompt word template... From basic template and prediction template Composition; specifically as follows:

[0019] ;

[0020] in, Representing the teacher model; This represents the i-th sample; This indicates the template for the first prompt word; This represents the sentiment reasoning chain inference content of the generated i-th sample; This represents the predicted sentiment label for the i-th sample;

[0021] Step S1.3: For the predicted sentiment label in step S1.2, when At that time, the sample The sample is considered the one that corresponds to the correct predicted sentiment label. Included in the first real-label distillation dataset :

[0022] ;

[0023] in, This represents the i-th sample; This represents the sentiment reasoning chain inference content of the i-th sample generated in step S1.2; This represents the true sentiment label of the i-th sample; This represents the predicted sentiment label for the i-th sample; This represents the number of samples in the first true label distillation dataset;

[0024] Step S1.4: For the predicted sentiment label in step S1.2, when At that time, the sample Samples that are considered as incorrect predicted sentiment labels are input with the true sentiment label. And based on the second prompt word template Regenerate the predicted sentiment labels and regenerate the sentiment reasoning content that concludes with the correct predicted sentiment labels.

[0025] ;

[0026] in, Representing the teacher model; This represents the i-th sample; This indicates a template for the second cue word; This represents the sentiment reasoning chain inference content of the regenerated i-th sample; This represents the predicted sentiment label for the regenerated i-th sample;

[0027] Step S1.5: For the regenerated predicted sentiment label from step S1.4, when At that time, the sample The samples are considered as the correctly predicted sentiment labels of the regenerated samples. Incorporating the second real-label distillation dataset ;

[0028] Step S1.6: Distill the first true label dataset Second True Label Distillation Dataset Merge to form the real label distillation dataset :

[0029] ;

[0030] in, This represents a real-label distillation dataset; This represents the first true label distillation dataset; This represents the second true label distillation dataset;

[0031] Step S1.7: Teacher Model If the output is invalid, repeat steps S1.2-S1.6 to perform a retry operation until a valid output is obtained or the preset retry limit is reached. ,in, This indicates that the prompt word template sent to the teacher model is taken from the first prompt word template. Or second prompt word template When a retry occurs in step S1.2 When a retry occurs in step S1.4 The retry function is defined as follows:

[0032] ;

[0033] in, Let i represent the i-th sample. This represents the retry function. Indicates a prompt word template, This indicates that the teacher model is applied to the i-th sample. Based on prompt word template A single call; Indicates the current number of retries. This indicates the preset retry limit. This indicates that the corresponding sample will be discarded and will no longer be included in the real label distillation dataset.

[0034] As a preferred technical solution of the present invention: In step S1.2, the teacher model A multimodal large model is used to generate sentiment reasoning chain inference content and predicted sentiment labels for each sample in the dataset, with a short chain to long chain mixing ratio of 4:1.

[0035] As a preferred technical solution of the present invention, the specific steps of step S2 are as follows:

[0036] Step S2.1: Construct the teaching assistant model The training and inference process of the teaching assistant model follows the prompt word template of the teacher model, along with the corresponding total loss function. A LoRA fine-tuning strategy is employed for the teaching assistant model. Multi-task learning training is conducted, which includes the emotion thought chain generation task and the emotion classification task. During the training phase of the two tasks, the teaching assistant model is trained using an autoregressive language modeling loss function.

[0037] ;

[0038] in, It's a hyperparameter; It is the loss function for the task of generating emotional thought chains; It is the loss function for the sentiment classification task; It is the total loss function of the teaching assistant model;

[0039] Step S2.2: Utilize the trained teaching assistant model Distillation dataset of real labels Inference is made from the samples in the teaching assistant model. Extracting unnormalized logits vectors from the output layer Through temperature coefficient The scaling Softmax function normalization process yields the soft label probability distribution:

[0040] ;

[0041] in, Indicates from the teaching assistant model The unnormalized logits vector extracted from the output layer is the first... One element; Indicates from the teaching assistant model The unnormalized logits vector extracted from the output layer is the first... One element; Indicates the temperature coefficient; Indicates the first The probability distribution of soft labels for each element;

[0042] Constructing a soft-label distillation dataset :

[0043] ;

[0044] in, This represents the i-th sample; Let represent the soft label probability distribution of the i-th sample; This represents the true sentiment label of the i-th sample; This represents the predicted sentiment label of the teaching assistant model for the i-th sample; This indicates the number of samples in the soft-label distillation dataset.

[0045] As a preferred embodiment of the present invention: In step S2.1, the loss function for both the emotion thought chain generation task and the emotion classification task adopts the sequence generation loss function, which is as follows:

[0046] ;

[0047] in, Indicates the batch size; Indicates the first The sequence length of each sample; Representing the teaching assistant model Before Given the position label sequence, the first Generate predicted sentiment tags for each location. The conditional probability; Indicates the first The first sample Predicted sentiment tags for each location; Indicates the first Before each sample Predicted sentiment label sequences for each location; Representing the teaching assistant model For the Sample The output; Indicates an indicator function, when The value is 1 if it is true, and 0 otherwise. This represents the sequence generation loss function.

[0048] As a preferred technical solution of the present invention, the specific steps of step S3 are as follows:

[0049] Step S3.1: For the student model Construct a soft-label loss function and distill the dataset based on the soft-label loss function. For student models Make fine adjustments, as shown in the following formula:

[0050] ;

[0051] in, This represents the probability distribution of the soft labels output by the teaching assistant model; This represents the probability distribution of the student model's output; and These represent the teaching assistant model and the student model in the first... Probability values ​​for each category; The soft-label loss function represents the student model;

[0052] Step S3.2: For the student model Construct a loss function for learning real labels and distill the dataset based on real labels. For student models Make fine adjustments, as shown in the following formula:

[0053] ;

[0054] in, This represents the true sentiment label of the i-th sample; This represents the predicted sentiment label of the student model for the i-th sample; This represents the true label learning loss function of the student model;

[0055] Step S3.3: Construct the overall soft-label learning loss function and the overall true-label learning loss function for the student model, further construct the total loss function for the student model, and train the student model based on the true-label distillation dataset:

[0056] ;

[0057] ;

[0058] ;

[0059] in, It's a hyperparameter; It is the loss function of the student model for the mind chain generation task on the soft-label distillation dataset; It is the loss function of the student model for the sentiment classification task on the soft-label distillation dataset; It is the loss function of the student model on the mind chain generation task on the real label distillation dataset; It is the loss function of the student model on the sentiment classification task on the real label distillation dataset; It is the soft-label learning loss function for the entire student model; It is the overall true label learning loss function of the student model; It is the total loss function of the student model.

[0060] The present invention also designs a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the aforementioned sentiment analysis method based on sentiment thought chain reasoning enhancement and knowledge distillation.

[0061] The present invention also designs a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned sentiment analysis method based on sentiment thought chain reasoning enhancement and knowledge distillation.

[0062] Beneficial effects: Compared with the prior art, the advantages of the present invention include:

[0063] Through a two-step progressive emotional thought chain generation mechanism, an effective balance can be achieved between reasoning depth and sample diversity, improving the quality and scale of the distillation dataset.

[0064] By introducing a teaching assistant model to generate soft tags, the problem of the teacher model being unable to obtain soft tags due to its large number of parameters and inability to be deployed locally is effectively compensated, and the ability of the lightweight model to effectively transfer the sentiment reasoning ability of the large model is realized.

[0065] By employing a joint learning strategy of real labels and soft labels, a lightweight student model can acquire sentiment reasoning capabilities without redesigning the network structure, significantly improving the performance of multimodal aspect-level sentiment analysis. Attached Figure Description

[0066] Figure 1 This is a flowchart of a sentiment analysis method based on sentiment-based reasoning enhancement and knowledge distillation, according to an embodiment of the present invention.

[0067] Figure 2 This is a framework diagram of a sentiment analysis method based on sentiment thinking chain reasoning enhancement and knowledge distillation provided by an embodiment of the present invention;

[0068] Figure 3 This is a schematic diagram of a multi-task learning and training framework for a teaching assistant model provided in an embodiment of the present invention. Detailed Implementation

[0069] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0070] This invention provides a sentiment analysis method based on sentiment-based reasoning enhancement and knowledge distillation, referring to... Figure 1 , Figure 2 Perform the following steps S1-S4 to complete the sentiment analysis of the text and image content:

[0071] Step S1: Collect multiple samples containing text and related images, and label them with corresponding real sentiment tags; based on the real sentiment tags of the samples, use the teacher model to perform inference, generate sentiment thought chain inference content and predicted sentiment tags, and include the samples corresponding to the correct predicted sentiment tags into the first real label distillation dataset; for the samples corresponding to the incorrect predicted sentiment tags, regenerate the predicted sentiment tags based on the real sentiment tags, and include the samples corresponding to the regenerated correct predicted sentiment tags into the second real label distillation dataset. Merge the first real label distillation dataset and the second real label distillation dataset to form the real label distillation dataset.

[0072] The specific steps of step S1 are as follows:

[0073] Step S1.1: Extract the i-th sample text Related images and aspect word sequence The aspect word sequence The text is The subsequence for the i-th sample Pre-label the corresponding real emotional tags The categories include positive, negative, or neutral; based on the sample. and the corresponding text Related images Aspect word sequence Authentic emotional tags ,by Data sets are constructed in a formal manner;

[0074] Step S1.2: Based on samples True emotional tags and first prompt word template Using a teacher model The process involves reasoning to generate an emotional thought chain and predicting emotional tags, wherein the first prompt word template... From basic template and prediction template Composition; specifically as follows:

[0075] ;

[0076] in, Representing the teacher model; This represents the i-th sample; This indicates the template for the first prompt word; This represents the sentiment reasoning chain inference content of the generated i-th sample; This represents the predicted sentiment label for the i-th sample;

[0077] In the embodiment, the teacher model A multimodal large-scale model with medium to large parameter counts is used as the teacher model. Since the teacher model's parameter count is too large to be deployed locally, the generation of thought chain data is accomplished by calling a third-party API. In this embodiment, either Qwen3-VL-32B (32 billion parameters) or Qwen3-VL-235B (235 billion parameters) is selected. Short and long thought chains are mixed in a 4:1 ratio to generate sentiment thought chain inference content and predicted sentiment labels for each sample in the dataset. Specifically, for every four samples in the dataset, one short thought chain is generated, and for every one sample, one long thought chain is generated, thus achieving an effective balance between inference depth and sample diversity.

[0078] Basic Template Includes pre-defined task descriptions, core principles, sentiment definitions, sample information, and inference formats; prediction templates. The study comprises four parts: multimodal analysis, key evidence, final inference conclusion, and prediction results. Multimodal analysis uses a combined text-image analysis of aspect terms from multiple dimensions, including visual features, aesthetic attributes, political and historical background, and social context. The key evidence section deeply integrates image and text information to extract core evidence determining the sentiment tendency of aspect terms, ultimately forming a logically coherent and accurately judged sentiment inference prediction conclusion.

[0079] The processing flow for the above steps is as follows: Sample With the first prompt word template Jointly sent to teacher model That is, a multimodal large model performs reasoning, and the teacher model outputs the reasoning content of the emotional thinking chain. and predicting sentiment tags The emotional thought chain reasoning content describes the intermediate reasoning process of the teacher model in making emotional judgments, and the predicted emotional label is the emotional polarity result inferred by the teacher model based on the input information.

[0080] Step S1.3: For the predicted sentiment label in step S1.2, when At that time, the sample The sample is considered the one that corresponds to the correct predicted sentiment label. Included in the first real-label distillation dataset :

[0081] ;

[0082] in, This represents the i-th sample; This represents the sentiment reasoning chain inference content of the i-th sample generated in step S1.2; This represents the true sentiment label of the i-th sample; This represents the predicted sentiment label for the i-th sample; This represents the number of samples in the first true label distillation dataset;

[0083] The processing flow of the above steps in the model is as follows: The predicted sentiment label output from step S1.2... With genuine emotion tags Perform comparison and screening, and retain only the samples that are correctly predicted. ), and compare it with the corresponding sample And emotional thought chain reasoning content Combine and construct the first real label distillation dataset This is used for subsequent model training.

[0084] Step S1.4: For the predicted sentiment label in step S1.2, when At that time, the sample Samples that are considered as incorrect predicted sentiment labels are input with the true sentiment label. And based on the second prompt word template Regenerate the predicted sentiment labels and regenerate the sentiment reasoning chain content that concludes with the correct predicted sentiment labels, thereby correcting the reasoning process of incorrect predictions.

[0085] ;

[0086] in, Representing the teacher model; This represents the i-th sample; This indicates a template for the second cue word; This represents the sentiment reasoning chain inference content of the regenerated i-th sample; This represents the predicted sentiment label for the regenerated i-th sample;

[0087] Step S1.5: For the regenerated predicted sentiment label from step S1.4, when At that time, the sample The samples are considered as the correctly predicted sentiment labels of the regenerated samples. Incorporating the second real-label distillation dataset ;

[0088] Step S1.6: Distill the first true label dataset Second True Label Distillation Dataset Merge to form the real label distillation dataset :

[0089] ;

[0090] in, This represents a real-label distillation dataset; This represents the first true label distillation dataset; This represents the second true label distillation dataset;

[0091] Step S1.7: Teacher Model If the output is invalid (such as an incomplete output structure or a failed call), repeat steps S1.2-S1.6 to perform a retry operation until a valid output is obtained or the preset retry limit is reached. ,in, This indicates that the prompt word template sent to the teacher model is taken from the first prompt word template. Or second prompt word template When a retry occurs in step S1.2 When a retry occurs in step S1.4 The retry function is defined as follows:

[0092] ;

[0093] in, Let i represent the i-th sample. This represents the retry function. Indicates a prompt word template, This indicates that the teacher model is applied to the i-th sample. Based on prompt word template For a single call, "output valid" means that the model returns a result that is structurally complete and contains sentiment reasoning content and predicted sentiment labels; Indicates the current number of retries. This indicates a preset retry limit. In the embodiment, it is set... , This indicates that the corresponding sample will be discarded and will no longer be included in the real label distillation dataset.

[0094] Step S2: Based on the teacher model, construct the teaching assistant model and the corresponding total loss function, train the teaching assistant model through multi-task learning, use the trained teaching assistant model to infer the samples in the real label distillation dataset, and construct the soft label distillation dataset based on the inference results.

[0095] Reference Figure 3 The specific steps of step S2 are as follows:

[0096] Step S2.1: Construct the teaching assistant model The corresponding total loss function is used. In this embodiment, the teaching assistant model is Qwen3-VL-8B (8 billion parameters), which jointly performs the emotion thought chain generation task and the emotion classification task. The training and inference process of the teaching assistant model follows the prompt word template of the teacher model; the LoRA fine-tuning strategy is used to fine-tune the teaching assistant model. Multi-task learning training is conducted, which includes the emotion thought chain generation task and the emotion classification task. During the training phase of the two tasks, the teaching assistant model is trained using an autoregressive language modeling loss function.

[0097] ;

[0098] in, It's a hyperparameter; It is the loss function for the task of generating emotional thought chains; It is the loss function for the sentiment classification task; It is the total loss function of the teaching assistant model;

[0099] Loss function for generating emotional thought chains Loss function for sentiment classification task According to hyperparameters and The total loss function of the teaching assistant model is formed by proportionally weighted summation. Used for backpropagation to update model parameters.

[0100] The loss function for both the emotion thought chain generation task and the emotion classification task adopts the sequence generation loss function, which is as follows:

[0101] ;

[0102] in, Indicates the batch size; Indicates the first The sequence length of each sample; Representing the teaching assistant model Before Given the position label sequence, the first Generate predicted sentiment tags for each location. The conditional probability; Indicates the first The first sample Predicted sentiment tags for each location; Indicates the first Before each sample Predicted sentiment label sequences for each location; Representing the teaching assistant model For the Sample The output; Indicates an indicator function, when The value is 1 if it is true, and 0 otherwise. This represents the sequence generation loss function.

[0103] Step S2.2: Utilize the trained teaching assistant model Distillation dataset of real labels Inference is made from the samples in the teaching assistant model. Extracting unnormalized logits vectors from the output layer Through temperature coefficient The scaling Softmax function normalization process yields the soft label probability distribution:

[0104] ;

[0105] in, Indicates from the teaching assistant model The unnormalized logits vector extracted from the output layer is the first... One element; Indicates from the teaching assistant model The unnormalized logits vector extracted from the output layer is the first... One element; It represents the temperature coefficient, used to adjust the smoothness of the distribution; Indicates the first The probability distribution of soft labels for each element;

[0106] Constructing a soft-label distillation dataset The decision logic used to guide student model learning from the teaching assistant model:

[0107] ;

[0108] in, This represents the i-th sample; Let represent the soft label probability distribution of the i-th sample; This represents the true sentiment label of the i-th sample; This represents the predicted sentiment label of the teaching assistant model for the i-th sample; This indicates the number of samples in the soft-label distillation dataset.

[0109] Step S3: Construct a student model and the corresponding soft-label loss function and real-label learning loss function. Fine-tune the student model based on the soft-label distillation dataset and the real-label distillation dataset, respectively. Further construct the total loss function of the student model and train the student model based on the real-label distillation dataset.

[0110] The specific steps of step S3 are as follows:

[0111] Step S3.1: For the student model In this embodiment, Qwen3-VL-2B (2 billion parameters) is selected as a lightweight multimodal model, with fewer parameters than the teaching assistant model. (8 billion parameters). A soft-label loss function is constructed based on the KL (Kullback-Leibler) divergence loss, and the soft-label distillation dataset is used. For student models Make fine adjustments, as shown in the following formula:

[0112] ;

[0113] in, This represents the probability distribution of the soft labels output by the teaching assistant model; This represents the probability distribution of the student model's output; and These represent the teaching assistant model and the student model in the first... Probability values ​​for each category; The soft-label loss function represents the student model;

[0114] The processing flow of the above steps is as follows: Student Model Based on soft-label distillation dataset To learn, the soft label distribution output by the teaching assistant model is simultaneously input into each sample. and the student model's own prediction distribution The difference between the two distributions is calculated using KL divergence as the loss. This guides the student model to gradually approximate the probability output of the teaching assistant model.

[0115] Step S3.2: For the student model Construct a loss function for learning real labels and distill the dataset based on real labels. For student models Make fine adjustments, as shown in the following formula:

[0116] ;

[0117] in, This represents the true sentiment label of the i-th sample; This represents the predicted sentiment label of the student model for the i-th sample; This represents the true label learning loss function of the student model;

[0118] Step S3.3: Construct the overall soft-label learning loss function and the overall true-label learning loss function for the student model, further construct the total loss function for the student model, and train the student model based on the true-label distillation dataset:

[0119] The training parameters are set as shown in Table 1 below:

[0120] Table 1. Training parameter settings

[0121]

[0122] The loss function is set as follows:

[0123] ;

[0124] ;

[0125] ;

[0126] in, It's a hyperparameter; It is the loss function of the student model for the mind chain generation task on the soft-label distillation dataset; It is the loss function of the student model for the sentiment classification task on the soft-label distillation dataset; It is the loss function of the student model on the mind chain generation task on the real label distillation dataset; It is the loss function of the student model on the sentiment classification task on the real label distillation dataset; It is the soft-label learning loss function for the entire student model; It is the overall true label learning loss function of the student model; It is the total loss function of the student model.

[0127] The processing flow of the above steps is as follows: the student model is based on the soft-label distillation dataset. and the real label distillation dataset Learning is conducted, with soft label learning loss. Loss generated by thought chain and loss classified by sentiment and Weighted composition, true label learning loss It is also composed of the weighted average of the losses from the two tasks, and the total loss is obtained by summing the two. Used to update student model parameters, enabling the student model to simultaneously acquire sentiment reasoning and sentiment classification capabilities under joint training with soft label supervision and real label supervision.

[0128] Step S4: Deploy the trained student model in the target application environment to complete sentiment analysis of text and image content.

[0129] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the aforementioned sentiment analysis method based on sentiment thought chain reasoning enhancement and knowledge distillation.

[0130] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned sentiment analysis method based on sentiment thought chain reasoning enhancement and knowledge distillation.

[0131] The following is an application example of the present invention:

[0132] This invention validates the performance of the proposed method on two publicly available datasets, Twitter 2015 and Twitter 2017. These two datasets are widely used benchmark datasets in the field of multimodal aspect-level sentiment analysis. Each sample contains tweet text, related images, and a sequence of aspect words, where aspect words are subsequences of the text. Sentiment labels are assigned to one of three categories: positive, negative, or neutral. Specific dataset statistics are shown in Table 2 below.

[0133] Table 2. Dataset Statistics

[0134]

[0135] The "4276 / 4301" column in the effective training set column represents the ratio of the number of successful LLM responses to the size of the original dataset. In this embodiment of the invention, the training set and validation set are combined as training data for distillation learning, while the original test set is retained for model performance evaluation. Due to security restrictions imposed by third-party API platforms, inputs containing sensitive words may be rejected by the large model, therefore the effective training set is slightly smaller than the original dataset size.

[0136] Baseline Comparison: In this embodiment, the following model is selected as the baseline for comparing model performance.

[0137] (1) ESAFN: Proposes an entity-sensitive attention and fusion network that can effectively model intramodal and intermodal interactions.

[0138] (2) FITE: Introducing facial emotions in images as visual emotional cues, a method for converting images into emotional text based on face sensitivity is proposed.

[0139] (3) MPFIT: It enhances the interaction modeling between images and aspects, can capture deep interactions between input aspects, and introduces prompt tokens to improve fusion efficiency.

[0140] (4) CoolNet: Construct a cross-modal fine-grained alignment and fusion network, incorporate syntactic knowledge, and realize fine-grained interaction between text and graphics and visual graph modeling.

[0141] (5) BIPF: Combining noise filtering and rapid learning, it filters out irrelevant visual information and achieves efficient modal fusion through bidirectional interactive prompts.

[0142] (6) VLP-MABSA: A visual language pre-training framework designed for multimodal aspect-level sentiment analysis. It adopts a unified encoding and decoding architecture and adapts to three types of pre-training tasks to improve fine-grained alignment and recognition capabilities.

[0143] (7) ITRIN: Aligns and interacts features of different modalities, and uses image-text probability scores for final feature denoising and fusion.

[0144] (8) Atlantis: The model consists of three branches: text-visual alignment aspect-emotion extraction, emotion perception image aesthetic evaluation, and aesthetic perception JMASA.

[0145] The comparison results of model performance are shown in Table 3:

[0146] Table 3. Comparison of Model Performance

[0147]

[0148] In Table 3, Acc represents the accuracy and F1 represents the F1 score. As shown in Table 3, the method proposed in this invention is generally superior to the performance of all baseline models on two public datasets. The teaching assistant model performs slightly better than the student model, but the number of parameters in the student model is only 1 / 4 of that in the teaching assistant model, thus verifying the effectiveness of this invention.

[0149] Where ECoTKD(ass) is the teaching assistant model. (Qwen3-VL-8B), after LoRA fine-tuning and multi-task learning training, was directly used for sentiment classification inference; ECoTKD(stu) is the student model. (Qwen3-VL-2B) is distilled from the teaching assistant model through joint learning of real labels and soft labels;

[0150] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for emotion analysis based on emotion thinking chain reasoning enhancement and knowledge distillation, characterized in that, Perform the following steps S1-S4 to complete the sentiment analysis of the text and image content: Step S1: Collect multiple samples containing text and related images, and label them with corresponding real sentiment tags; based on the real sentiment tags of the samples, use the teacher model to perform inference, generate sentiment thought chain inference content and predicted sentiment tags, and include the samples corresponding to the correct predicted sentiment tags into the first real label distillation dataset; for the samples corresponding to the incorrect predicted sentiment tags, regenerate the predicted sentiment tags based on the real sentiment tags, and include the samples corresponding to the regenerated correct predicted sentiment tags into the second real label distillation dataset. Merge the first real label distillation dataset and the second real label distillation dataset to form the real label distillation dataset. Step S2: Based on the teacher model, construct the teaching assistant model and the corresponding total loss function, train the teaching assistant model through multi-task learning, use the trained teaching assistant model to infer the samples in the real label distillation dataset, and construct the soft label distillation dataset based on the inference results. Step S3: Construct a student model and the corresponding soft-label loss function and real-label learning loss function. Fine-tune the student model based on the soft-label distillation dataset and the real-label distillation dataset, respectively. Further construct the total loss function of the student model and train the student model based on the real-label distillation dataset. Step S4: Deploy the trained student model in the target application environment to complete sentiment analysis of text and image content.

2. The sentiment analysis method based on sentiment-based reasoning enhancement and knowledge distillation according to claim 1, characterized in that, The specific steps of step S1 are as follows: Step S1.1: Extract the i-th sample text Related images and aspect word sequence The aspect word sequence The text is The subsequence for the i-th sample Pre-label the corresponding real emotional tags The categories include positive, negative, or neutral; based on the sample. and the corresponding text Related images Aspect word sequence Authentic emotional tags ,by Data sets are constructed in a formal manner; Step S1.2: Based on samples True emotional tags and first prompt word template Using a teacher model The process involves reasoning to generate an emotional thought chain and predicting emotional tags, wherein the first prompt word template... From basic template and prediction template Composition; specifically as follows: ； in, Representing the teacher model; This represents the i-th sample; This indicates the template for the first prompt word; This represents the sentiment reasoning chain inference content of the generated i-th sample; This represents the predicted sentiment label for the i-th sample; Step S1.3: For the predicted sentiment label in step S1.2, when At that time, the sample The sample is considered the one that corresponds to the correct predicted sentiment label. Included in the first real-label distillation dataset : ； in, This represents the i-th sample; This represents the sentiment reasoning chain inference content of the i-th sample generated in step S1.2; This represents the true sentiment label of the i-th sample; This represents the predicted sentiment label for the i-th sample; This represents the number of samples in the first true label distillation dataset; Step S1.4: For the predicted sentiment label in step S1.2, when At that time, the sample Samples that are considered as incorrect predicted sentiment labels are input with the true sentiment label. And based on the second prompt word template Regenerate the predicted sentiment labels and regenerate the sentiment reasoning content that concludes with the correct predicted sentiment labels. ； in, Representing the teacher model; This represents the i-th sample; This indicates a template for the second cue word; This represents the sentiment reasoning chain inference content of the regenerated i-th sample; This represents the predicted sentiment label for the regenerated i-th sample; Step S1.5: For the regenerated predicted sentiment label from step S1.4, when At that time, the sample The samples are considered as the correctly predicted sentiment labels of the regenerated samples. Incorporating the second real-label distillation dataset ; Step S1.6: Distill the first true label dataset Second True Label Distillation Dataset Merge to form the real label distillation dataset : ； in, This represents a real-label distillation dataset; This represents the first true label distillation dataset; This represents the second true label distillation dataset; Step S1.7: Teacher Model If the output is invalid, repeat steps S1.2-S1.6 to perform a retry operation until a valid output is obtained or the preset retry limit is reached. ,in, This indicates that the prompt word template sent to the teacher model is taken from the first prompt word template. Or second prompt word template When a retry occurs in step S1.2 When a retry occurs in step S1.4 The retry function is defined as follows: ； in, Let i represent the i-th sample. This represents the retry function. Indicates a prompt word template, This indicates that the teacher model is applied to the i-th sample. Based on prompt word template A single call; Indicates the current number of retries. This indicates the preset retry limit. This indicates that the corresponding sample will be discarded and will no longer be included in the real label distillation dataset.

3. The sentiment analysis method based on sentiment-based reasoning enhancement and knowledge distillation according to claim 2, characterized in that, In step S1.2, the teacher model A multimodal large model is used to generate sentiment reasoning chain inference content and predicted sentiment labels for each sample in the dataset, with a short chain to long chain mixing ratio of 4:

1.

4. The sentiment analysis method based on sentiment-based reasoning enhancement and knowledge distillation according to claim 2, characterized in that, The specific steps of step S2 are as follows: Step S2.1: Construct the teaching assistant model The training and inference process of the teaching assistant model follows the prompt word template of the teacher model, along with the corresponding total loss function. Using LoRA fine-tuning strategy for teaching assistant model Multi-task learning training is conducted, which includes the emotion thought chain generation task and the emotion classification task. During the training phase of the two tasks, the teaching assistant model is trained using an autoregressive language modeling loss function. ； in, It's a hyperparameter; It is the loss function for the task of generating emotional thought chains; It is the loss function for the sentiment classification task; It is the total loss function of the teaching assistant model; Step S2.2: Utilize the trained teaching assistant model Distillation dataset of real labels Inference is made from the samples in the teaching assistant model. Extracting unnormalized logits vectors from the output layer Through temperature coefficient The scaling Softmax function normalization process yields the soft label probability distribution: ； in, Indicates from the teaching assistant model The unnormalized logits vector extracted from the output layer is the first... One element; Indicates from the teaching assistant model The unnormalized logits vector extracted from the output layer is the first... One element; Indicates the temperature coefficient; Indicates the first The probability distribution of soft labels for each element; Constructing a soft-label distillation dataset : ； in, This represents the i-th sample; Let represent the soft label probability distribution of the i-th sample; This represents the true sentiment label of the i-th sample; This represents the predicted sentiment label of the teaching assistant model for the i-th sample; This indicates the number of samples in the soft-label distillation dataset.

5. The sentiment analysis method based on sentiment-based reasoning enhancement and knowledge distillation according to claim 4, characterized in that, In step S2.1, the loss function for both the emotion thought chain generation task and the emotion classification task adopts the sequence generation loss function, which is as follows: ； in, Indicates the batch size; Indicates the first The sequence length of each sample; Representing the teaching assistant model Before Given the position label sequence, the first Generate predicted sentiment tags for each location. The conditional probability; Indicates the first The first sample Predicted sentiment tags for each location; Indicates the first Before each sample Predicted sentiment label sequences for each location; Representing the teaching assistant model For the Sample The output; Indicates an indicator function, when The value is 1 if it is true, and 0 otherwise. This represents the sequence generation loss function.

6. The sentiment analysis method based on sentiment-based reasoning enhancement and knowledge distillation according to claim 5, characterized in that, The specific steps of step S3 are as follows: Step S3.1: For the student model Construct a soft-label loss function and distill the dataset based on the soft-label loss function. For student models Make fine adjustments, as shown in the following formula: ； in, This represents the probability distribution of the soft labels output by the teaching assistant model; This represents the probability distribution of the student model's output; and These represent the teaching assistant model and the student model in the first... Probability values ​​for each category; The soft-label loss function represents the student model; Step S3.2: For the student model Construct a loss function for learning real labels and distill the dataset based on real labels. For student models Make fine adjustments, as shown in the following formula: ； in, This represents the true sentiment label of the i-th sample; This represents the predicted sentiment label of the student model for the i-th sample; This represents the true label learning loss function of the student model; Step S3.3: Construct the overall soft-label learning loss function and the overall true-label learning loss function for the student model, further construct the total loss function for the student model, and train the student model based on the true-label distillation dataset: ； ； ； in, It's a hyperparameter; It is the loss function of the student model for the mind chain generation task on the soft-label distillation dataset; It is the loss function of the student model for the sentiment classification task on the soft-label distillation dataset; It is the loss function of the student model on the mind chain generation task on the real label distillation dataset; It is the loss function of the student model on the sentiment classification task on the real label distillation dataset; It is the soft-label learning loss function for the entire student model; It is the overall true label learning loss function of the student model; It is the total loss function of the student model.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements each step of the sentiment analysis method based on sentiment thought chain reasoning enhancement and knowledge distillation as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the sentiment analysis method based on sentiment thought chain reasoning enhancement and knowledge distillation as described in any one of claims 1 to 6.

Images