Sentiment analysis method, device and equipment based on knowledge distillation and passive domain adaptation

Abstract

The present invention provides a sentiment analysis method, device and equipment for passive domain adaptation based on knowledge distillation, which relates to the field of transfer learning. It includes: obtaining a source domain model including a feature extractor and a classifier; initializing the feature extractor of the bridge domain model with the model parameters of the source domain model feature extractor, using the model parameters of the source domain model classifier as the model parameters of the bridge domain model classifier, obtaining the bridge domain model to be trained as a student model, and using the source domain model as a teacher model to perform knowledge distillation learning to obtain the bridge domain model; randomly initializing the model parameters of the target domain model feature extractor, using the model parameters of the source domain model classifier as the model parameters of the target domain model classifier, obtaining the target domain model to be trained as a student model, and using the bridge domain model as a teacher model to perform knowledge distillation learning to obtain the target domain model, so as to perform sentiment classification prediction on the image data of the target domain, thereby improving the performance of the passive domain adaptation method in domain adaptation in sentiment data sets.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of transfer learning, and in particular to a sentiment analysis method, device and equipment based on knowledge distillation and passive domain adaptation. Background Art

[0002] In recent years, with the rapid development of photography technology and mobile devices, people are accustomed to posting pictures, videos and texts on social platforms to express their thoughts and feelings. This large-scale multimedia data plays an important role in various applications, such as depression detection and opinion mining. Among them, the visual emotion recognition task aims to identify human emotions triggered by images, that is, to analyze human emotional responses to different visual stimuli. Due to the subjectivity and ambiguity of emotional data, it is very difficult to manually annotate a reliable large-scale emotional dataset. In order to reduce the dependence of neural network models on labeled data, domain adaptation methods in transfer learning provide a solution: adapting the model trained on labeled source domain data to unlabeled target domain data.

[0003] However, the current related domain adaptation methods need to use source domain data in the adaptation process. For privacy reasons, the sentiment data of the source domain may not be accessible. The current passive domain adaptation methods that do not use source domain data in the adaptation process will affect the effect of domain adaptation when targeting visual emotion recognition tasks due to the perceptual subjectivity in the sentiment data. Summary of the invention

[0004] Based on the above technical problems, the embodiments of the present invention provide a sentiment analysis method, device and equipment with passive domain adaptation based on knowledge distillation, aiming to improve the performance of passive domain adaptation method in domain adaptation in sentiment data sets.

[0005] A first aspect of an embodiment of the present invention provides a sentiment analysis method based on knowledge distillation and passive domain adaptation, the method comprising:

[0006] Obtaining a pre-trained source domain model, where the source domain model is trained using source domain image data with emotion labels, and the source domain model includes a feature extractor and a classifier;

[0007] Initializing the feature extractor of the bridge domain model using the model parameters of the feature extractor of the source domain model, taking the model parameters of the classifier of the source domain model as the model parameters of the classifier of the bridge domain model and fixing them, thereby obtaining the bridge domain model to be trained;

[0008] Using the bridge domain model to be trained as a student model and the pre-trained source domain model as a teacher model, knowledge distillation learning is performed to obtain a trained bridge domain model;

[0009] The model parameters of the feature extractor of the target domain model are randomly initialized, and the model parameters of the classifier of the pre-trained source domain model are used as the model parameters of the classifier of the target domain model and fixed to obtain the target domain model to be trained;

[0010] The target domain model to be trained is used as a student model, and the trained bridge domain model is used as a teacher model to perform knowledge distillation learning to obtain a trained target domain model, wherein the loss function for training the target domain model at least includes an alignment loss function, and the alignment loss function is used to align the sentiment predictions of the bridge domain model and the target domain model;

[0011] Using the trained target domain model, sentiment classification prediction is performed on the image data of the target domain to obtain a sentiment analysis result.

[0012] A second aspect of an embodiment of the present invention provides a sentiment analysis device based on knowledge distillation and passive domain adaptation, the device comprising:

[0013] A first model acquisition module is used to obtain a pre-trained source domain model, where the source domain model is trained using source domain image data with emotion labels, and the source domain model includes a feature extractor and a classifier;

[0014] A second model acquisition module is used to initialize the feature extractor of the bridge domain model using the model parameters of the feature extractor of the source domain model, and to fix the model parameters of the classifier of the source domain model as the model parameters of the classifier of the bridge domain model, so as to obtain the bridge domain model to be trained;

[0015] A first model training module is used to perform knowledge distillation learning using the bridge domain model to be trained as a student model and the pre-trained source domain model as a teacher model to obtain a trained bridge domain model;

[0016] A third model acquisition module is used to randomly initialize the model parameters of the feature extractor of the target domain model, and use the model parameters of the classifier of the pre-trained source domain model as the model parameters of the classifier of the target domain model and fix them to obtain the target domain model to be trained;

[0017] A second model training module is used to perform knowledge distillation learning using the target domain model to be trained as a student model and the trained bridge domain model as a teacher model to obtain a trained target domain model, wherein the loss function for training the target domain model at least includes an alignment loss function, and the alignment loss function is used to align the sentiment predictions of the bridge domain model and the target domain model;

[0018] The sentiment analysis module is used to use the trained target domain model to perform sentiment classification prediction on the image data of the target domain to obtain a sentiment analysis result.

[0019] A third aspect of an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the sentiment analysis method based on knowledge distillation and passive domain adaptation as described in the first aspect of the embodiment of the present invention is implemented.

[0020] In the sentiment analysis method for passive domain adaptation based on knowledge distillation provided in an embodiment of the present invention, first, a pre-trained source domain model is obtained; secondly, the feature extractor of the bridge domain model is initialized using the model parameters of the feature extractor of the source domain model, and the model parameters of the classifier of the source domain model are used as the model parameters of the classifier of the bridge domain model and fixed to obtain the bridge domain model to be trained; then, the bridge domain model to be trained is used as a student model and the pre-trained source domain model is used as a teacher model to perform knowledge distillation learning to obtain a trained bridge domain model; then, the model parameters of the feature extractor of the target domain model are randomly initialized, and the model parameters of the classifier of the pre-trained source domain model are used as the model parameters of the classifier of the target domain model and fixed to obtain the target domain model to be trained; finally, the target domain model to be trained is used as a student model and the trained bridge domain model is used as a teacher model to perform knowledge distillation learning to obtain a trained target domain model; the trained target domain model is used to perform sentiment classification prediction on the image data of the target domain to obtain a sentiment analysis result. Through the sentiment analysis method of this embodiment, for passive domain adaptation of visual emotion recognition, a bridge domain model is introduced to enhance the consistency between the source domain and the target domain to solve the problem of uneven distribution of sentiment data. The target domain model is learned from scratch under the supervision of the bridge domain model, which helps to reduce overfitting of the source domain, and inter-domain sentiment alignment is performed between the bridge model and the target model to promote the connection between the bridge model and the target model, thereby improving the performance of domain adaptation of the passive domain adaptation method in the sentiment data set without using the source domain data, and the learned target domain features are more similar to the original structure of the target domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0022] Figure 1It is a schematic diagram of the performance of the passive domain adaptation method in the related art on the visual emotion recognition task;

[0023] Figure 2 is a flowchart of a sentiment analysis method based on knowledge distillation and passive domain adaptation according to an embodiment of the present invention;

[0024] Figure 3 is a schematic diagram showing a comparison between a traditional passive domain adaptation method and a passive emotion domain adaptation method based on knowledge distillation according to an embodiment of the present invention;

[0025] Figure 4 It is a structural block diagram of a sentiment analysis device based on knowledge distillation and passive domain adaptation provided by an embodiment of the present invention;

[0026] Figure 5 It is a schematic diagram of an electronic device shown in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] At present, for the task of visual emotion recognition, people have proposed technologies based on deep neural networks to improve the performance of emotion recognition. Related technologies mainly use supervised methods and require sufficient emotion annotation labels. However, due to the subjectivity, complexity and ambiguity of emotions, it is very challenging to label reliable large-scale datasets for visual emotion recognition tasks. In order to reduce the burden of labeling, people have begun to pay attention to unsupervised domain adaptation, so that models trained on labeled source domain data can better generalize on unlabeled target data.

[0029] Existing unsupervised domain adaptation methods usually require access to source data during the adaptation process. Considering privacy and security issues, passive domain adaptation methods propose to not use source data during the adaptation process. However, while previous passive domain adaptation techniques are effective for standard classification tasks, when directly applied to visual emotion recognition tasks, they encounter challenges specific to this task and their performance degrades because the perceptual subjectivity in the emotion data affects the effectiveness of domain adaptation.

[0030] To better illustrate this problem, Figure 1 As shown, Figure 1 This is a schematic diagram of the performance of the passive domain adaptation method in the related art on the visual emotion recognition task. Figure 1(a) reflects the ambiguity of the emotion dataset: ResNet-101 models with the same number of categories were trained on the visual emotion recognition dataset EmoSet and the standard classification dataset Clipart. Figure 1 The upper part of (a) shows the performance of the visual emotion recognition dataset EmoSet directly adapted on the ResNet-101 model. Figure 1 The lower part of (a) shows the performance of the Offce-Home dataset, i.e. the standard classification dataset Clipart, directly adapted to the ResNet-101 model. The results show that the performance of the ResNet-101 model on the Clipart dataset far exceeds that on the EmoSet. The main reason for this difference is that the feature distribution of the emotional context in the domain is relatively vague.

[0031] In addition, from Figure 1 As can be seen from (b) in the figure, the traditional passive domain adaptation method has considerable difficulties in the visual emotion recognition task: when directly adapting from the source domain dataset to the target dataset EmoSet->FI and Clipart->Real World, without further improvement, the performance of the standard classification model will greatly exceed the visual emotion recognition model, which indicates that the inter-domain correlation of the emotion dataset is lower than that of the standard classification task.

[0032] It can be seen that related passive domain adaptation methods (such as pseudo-labeling, contrastive learning, and class prototypes) provide possible solutions, but it is often difficult to strike a balance between maintaining the source-invariant domain knowledge and reducing the noise of source-specific knowledge. This is because in self-supervised methods, the lack of unique category features often leads to classification errors. Taking pseudo-labeling methods as an example, Figure 1 As shown in (b) in Figure 2, SHOT, a typical passive domain adaptation method, is used to generate pseudo labels for EmoSet->FI and Real World->Clipart. It can be seen that the classification scores of the emotional pseudo labels are relatively low, which increases the risk of inaccurate pseudo labels and may lead to confirmation bias due to error accumulation.

[0033] Therefore, in order to at least partially solve one or more of the above-mentioned problems and other potential problems, an embodiment of the present invention proposes a sentiment analysis method based on knowledge distillation and passive domain adaptation. In this method, for passive domain adaptation of visual emotion recognition, it is necessary to simultaneously process two tasks, namely, visual emotion recognition and passive domain adaptation. The bridge domain model is introduced to enhance the consistency between the source domain and the target domain to solve the problem of uneven distribution of sentiment data. The target domain model is learned from scratch under the supervision of the bridge domain model, which helps to reduce overfitting of the source domain, and inter-domain sentiment alignment is performed between the bridge model and the target model to promote the connection between the bridge model and the target model, thereby improving the performance of the passive domain adaptation method in domain adaptation in sentiment data sets, and the learned target domain features are more similar to the original structure of the target domain.

[0034] Please refer to Figure 2 , Figure 2 FIG. 1 is a flowchart of a method for sentiment analysis based on knowledge distillation and passive domain adaptation according to an embodiment of the present invention. Figure 2 As shown, the sentiment analysis method based on knowledge distillation and passive domain adaptation provided in this embodiment includes at least the following steps:

[0035] Step S11: obtaining a pre-trained source domain model, wherein the source domain model is trained using source domain image data with emotion labels, and the source domain model includes a feature extractor and a classifier.

[0036] In this embodiment, a pre-trained source domain model can be obtained first, and the source domain module includes: a feature extractor and a classifier. The pre-trained source domain model in this embodiment is a sentiment classification model trained using source domain image data with sentiment labels, wherein the source domain image data with sentiment labels is source domain image data carrying sentiment labels.

[0037] Step S12: Initialize the feature extractor of the bridge domain model using the model parameters of the feature extractor of the source domain model, use the model parameters of the classifier of the source domain model as the model parameters of the classifier of the bridge domain model and fix them, and obtain the bridge domain model to be trained.

[0038] This embodiment proposes a bridge domain model to protect the source knowledge and bridge the gap between the source domain and the target domain. In order to better retain the source knowledge of the source domain model, the bridge domain model inherits parameters from the pre-trained source domain model. Since it is hoped that the bridge domain model can continue to learn the target domain, the feature extractor of the bridge domain model will be updated, while the classifier of the bridge domain model remains unchanged. Therefore, this embodiment uses the model parameters of the feature extractor of the source domain model to initialize the feature extractor of the bridge domain model, and uses the model parameters of the classifier of the source domain model as the model parameters of the classifier of the bridge domain model and fixes them to obtain the bridge domain model to be trained.

[0039] Step S13: Using the bridge domain model to be trained as a student model and the pre-trained source domain model as a teacher model, knowledge distillation learning is performed to obtain a trained bridge domain model.

[0040] In this embodiment, after obtaining the bridge domain model to be trained, the bridge domain model to be trained is used as the student model, and the pre-trained source domain model is used as the teacher model. The bridge domain model to be trained is trained through knowledge distillation learning to obtain a trained bridge domain model.

[0041] Step S14: randomly initialize the model parameters of the feature extractor of the target domain model, use the model parameters of the classifier of the pre-trained source domain model as the model parameters of the classifier of the target domain model and fix them, so as to obtain the target domain model to be trained.

[0042] The target domain model in this embodiment needs to be learned from scratch under the supervision of the bridge domain model. Therefore, the model parameters of the feature extractor of the target domain model in this embodiment do not inherit the model parameters of the feature extractor of the bridge domain model, but learn the target domain feature structure from scratch, that is, the model parameters of the feature extractor of the target domain model are randomly initialized to help reduce the overfitting of the target model to the source domain. In addition, the parameters of the classifier in this embodiment are always fixed: the model parameters of the classifier of the pre-trained source domain model are used as the model parameters of the classifier of the target domain model and fixed to obtain the target domain model to be trained.

[0043] Step S15: Using the target domain model to be trained as a student model and the trained bridge domain model as a teacher model, knowledge distillation learning is performed to obtain a trained target domain model, and the loss function for training the target domain model at least includes an alignment loss function, which is used to align the sentiment predictions of the bridge domain model and the target domain model.

[0044] In this embodiment, after obtaining the target domain model to be trained, the target domain model to be trained is used as the student model, and the trained bridge domain model is used as the teacher model. The target domain model to be trained is trained through knowledge distillation learning to obtain the trained target domain model. Among them, this embodiment proposes inter-domain sentiment alignment when training the target domain model to promote the connection between the bridge domain model and the target domain model. Specifically, the loss function of training the target domain model at least includes an alignment loss function, and the alignment loss function is used to align the sentiment predictions of the bridge domain model and the target domain model.

[0045] This is because the related traditional passive domain adaptation methods are based on the pre-trained source domain model weights, and the feature extractor of the source domain model is fine-tuned according to the target domain data to bridge the emotional gap between different emotional domains. However, the large amount of prior knowledge imparted by the source domain model may limit the representation of target domain features, thereby hindering the generalization ability of the bridge model. To overcome this problem, the present embodiment gives the target domain model enough degrees of freedom to allow it to explore the target feature space from scratch and identify new features. In an optional embodiment, the present embodiment uses a progressive emotional spontaneous fractionation technique to train the target domain model.

[0046] Furthermore, since the target domain model of this embodiment is trained from scratch, the bridge domain model uses past predictions to improve supervision throughout the training process. Noting that the bridge domain model contains sentiment information related to both the source domain and the target domain, the embodiment proposes an alignment method to constrain the model so that the sentiment predictions of the bridge domain model and the target domain model are aligned, thereby forcing the target domain model to learn the semantics of the sentiment context.

[0047] Step S16: using the trained target domain model, perform sentiment classification prediction on the image data of the target domain to obtain a sentiment analysis result.

[0048] In this embodiment, after obtaining the trained target domain model, visual emotion recognition can be performed through the trained target domain model: the trained target domain model is used to perform emotion classification prediction on the image data of the target domain to obtain the emotion analysis results corresponding to the image data of the target domain.

[0049] In this embodiment, for passive domain adaptation of visual emotion recognition, a bridge domain model is introduced to enhance the consistency between the source domain and the target domain to solve the problem of uneven distribution of emotion data. The target domain model is learned from scratch under the supervision of the bridge domain model, which helps to reduce overfitting of the source domain, and inter-domain emotion alignment is performed between the bridge model and the target model to promote the connection between the bridge model and the target model, thereby improving the performance of the passive domain adaptation method in domain adaptation in emotion datasets, and the learned target domain features are more similar to the original structure of the target domain.

[0050] In combination with the above embodiments, in one implementation, the present invention further provides a sentiment analysis method based on knowledge distillation and passive domain adaptation. In the method, the above step S13 may specifically include step S21 and step S22:

[0051] Step S21: inputting the target domain image data without emotion labels into the source domain model to obtain a first pseudo emotion label of the target domain image data without emotion labels.

[0052] In this embodiment, the sample data used to train the bridge domain model is target domain image data without emotion labels, and the target domain image data without emotion labels is target domain image data without emotion labels. In the absence of target emotion labels, distillation learning cannot alleviate the disturbance caused by label mismatch. Based on this, this embodiment uses a pre-trained source domain model to generate pseudo labels. Specifically, the target domain image data without emotion labels is input into the pre-trained source domain model to obtain the first pseudo emotion label of the target domain image data without emotion labels, wherein the first pseudo emotion label of this embodiment refers to the pseudo emotion label generated by the source domain model.

[0053] Step S22: input the target domain image data without emotion labels into the bridge domain model to be trained, and based on the first pseudo emotion label of the target domain image data without emotion labels, use the exponential moving average strategy EMA to update the model parameters of the feature extractor of the bridge domain model to obtain the trained bridge domain model.

[0054] In this embodiment, after obtaining the first pseudo emotion label of the target domain image data without emotion label, the target domain image data without emotion label is used as the training sample of the bridge domain model to be trained, and the first pseudo emotion label of the target domain image data without emotion label is used as the label of the bridge domain model to be trained, and the model parameters of the feature extractor of the bridge domain model are updated. Specifically, the target domain image data without emotion label is input into the bridge domain model to be trained, and based on the first pseudo emotion label of the target domain image data without emotion label, the model parameters of the feature extractor of the bridge domain model are updated using the exponential moving average strategy EMA to obtain a trained bridge domain model to align the emotion data from different fields. For example, when training the bridge domain model, the training of the bridge domain model may be stopped when the specified number of iterations is reached, and the trained bridge domain model may be obtained. For example, the specified number of iterations may be 20 epochs, and so on.

[0055] In this embodiment, the biggest challenge in training the bridge domain model is to mine domain-invariant knowledge and reduce domain-specific information between domains. In order to robustly refine the source domain model knowledge, the bridge domain model of this embodiment uses an exponential moving average strategy (EMA) for parameter update.

[0056] In combination with the above embodiments, in one implementation, the present invention further provides a sentiment analysis method based on knowledge distillation and passive domain adaptation. In this embodiment, the loss function for training the source domain model includes at least: a first loss function, wherein the first loss function is used to calculate the loss between the sentiment prediction result obtained by the source domain model based on the source domain image data with sentiment labels and the sentiment label corresponding to the source domain image data with sentiment labels. In this embodiment, the training of the pre-trained source domain model is to input the source domain image data with sentiment labels into the source domain model to be trained, and obtain the sentiment prediction result corresponding to the source domain image data with sentiment labels, and then calculate the first loss function based on the sentiment prediction result and the sentiment label corresponding to the source domain image data with sentiment labels to obtain the first loss, so as to adjust the model parameters of the source domain model to be trained based on the first loss until the pre-trained source domain model is obtained.

[0057] And, in this embodiment, the loss function for training the bridge domain model includes at least: a second loss function and a first distillation loss function. Among them, the second loss function is used to calculate the loss between the emotion prediction result obtained by the bridge domain model based on the target domain image data without emotion labels and the first pseudo emotion label of the target domain image data without emotion labels. In this embodiment, for the training of the bridge domain model, the target domain image data without emotion labels is input into the bridge domain model to be trained to obtain the emotion prediction result corresponding to the target domain image data without emotion labels, and then the second loss function is calculated based on the emotion prediction result corresponding to the target domain image data without emotion labels and the first pseudo emotion label corresponding to the target domain image data without emotion labels to obtain the second loss.

[0058] The first distillation loss function is used to calculate the KL divergence loss between the distribution of the emotion prediction results obtained by the bridge domain model based on the target domain image data without emotion labels and the distribution of the emotion prediction results obtained by the source domain model based on the target domain image data without emotion labels. In this embodiment, for the training of the bridge domain model, the target domain image data without emotion labels will also be input into the trained source domain model to obtain the emotion prediction results obtained by the source domain model based on the target domain image data without emotion labels, and then the first distillation loss function is calculated based on the distribution of the emotion prediction results obtained by the bridge domain model for the target domain image data without emotion labels and the distribution of the emotion prediction results obtained by the source domain model for the target domain image data without emotion labels, that is, the KL divergence loss is calculated to obtain the first distillation loss. Therefore, based at least on the second loss and the first distillation loss, the model parameters of the bridge domain model to be trained are adjusted until the trained bridge domain model is obtained.

[0059] In combination with the above embodiments, the present invention further provides a sentiment analysis method based on knowledge distillation and passive domain adaptation. In the method, the above step S15 of "taking the target domain model to be trained as a student model and the trained bridge domain model as a teacher model to perform knowledge distillation learning to obtain a trained target domain model" may specifically include steps S31 and S32:

[0060] Step S31: input the target domain image data without emotion labels into the trained bridge domain model to obtain a second pseudo emotion label of the target domain image data without emotion labels.

[0061] In this embodiment, the sample data used to train the target domain model is still the target domain image data without emotion labels. Similarly, in the absence of target emotion labels, distillation learning cannot alleviate the disturbance caused by label mismatch. Based on this, this embodiment uses the trained bridge domain model to generate pseudo labels. Specifically, the target domain image data without emotion labels is input into the trained bridge domain model to obtain the second pseudo emotion label corresponding to the target domain image data without emotion labels. The second pseudo emotion label in this embodiment refers to the pseudo emotion label generated by the bridge domain model.

[0062] Step S32: inputting the target domain image data without emotion labels into the target domain model to be trained, and based on the second pseudo emotion label of the target domain image data without emotion labels, updating the model parameters of the feature extractor of the target domain model to obtain the trained target domain model.

[0063] In this embodiment, after obtaining the second pseudo emotion label of the target domain image data without emotion label, the target domain image data without emotion label is used as the training sample of the target domain model to be trained, and the second pseudo emotion label of the target domain image data without emotion label is used as the label of the target domain model to be trained, and the model parameters of the feature extractor of the target domain model are updated. Specifically, the target domain image data without emotion label is input into the target domain model to be trained, and based on the second pseudo emotion label of the target domain image data without emotion label, the model parameters of the feature extractor of the target domain model are updated to obtain the trained target domain model. The average historical prediction of the bridge model (i.e., the second pseudo emotion label) provides guidance for alignment. Under the guidance of the bridge model and the re-learning of the target domain model, the deviation of the source domain training data will be reduced, so that the model can better focus on the emotional expression characteristics of the target domain.

[0064] In combination with the above embodiments, in one implementation, the present invention further provides a sentiment analysis method based on knowledge distillation and passive domain adaptation. In this embodiment, the loss function of training the target domain model includes, in addition to the alignment loss function, information maximization loss. The alignment loss function includes at least a third loss function and a second distillation loss function.

[0065] The third loss function is used to calculate the loss between the emotion prediction result obtained by the target domain model based on the target domain image data without emotion labels and the second pseudo emotion label of the target domain image data without emotion labels. In this embodiment, the training of the target domain model is to input the target domain image data without emotion labels into the target domain model to be trained to obtain the emotion prediction result corresponding to the target domain image data without emotion labels, and then calculate the third loss function based on the emotion prediction result and the second pseudo emotion label corresponding to the target domain image data without emotion labels to obtain the third loss.

[0066] The second distillation loss function is used to calculate the KL divergence loss between the distribution of the emotion prediction results obtained by the bridge domain model based on the target domain image data without emotion labels and the distribution of the emotion prediction results obtained by the target domain model based on the target domain image data without emotion labels. In this embodiment, for the training of the target domain model, the target domain image data without emotion labels will also be input into the trained bridge domain model to obtain the emotion prediction results obtained by the bridge domain model based on the target domain image data without emotion labels, and then the second distillation loss function is calculated based on the distribution of the emotion prediction results obtained by the bridge domain model for the target domain image data without emotion labels and the distribution of the emotion prediction results obtained by the target domain model for the target domain image data without emotion labels, that is, the KL divergence loss is calculated to obtain the second distillation loss.

[0067] The information maximization loss of this embodiment is used to calculate the prediction confidence maximization loss of each emotion category predicted by the target domain model for the target domain image data without emotion labels. The information maximization loss is based on two assumptions: first, it is assumed that a well-trained feature extractor of the target domain model can extract features that allow the classifier to generate high confidence; second, it is assumed that the emotion prediction results corresponding to the target domain image data without emotion labels should be evenly distributed in each category, rather than concentrated in a few categories.

[0068] For example, the information maximization loss of this embodiment is Can be: ;

[0069] in, ; To calculate the expectation; represents the target domain sample, k represents the category label, K represents the label space, represents the probability of category k.

[0070] Based on this, this embodiment adjusts the model parameters of the target domain model to be trained (the feature extractor of the target domain model) based on the third loss, the second distillation loss function and the information maximization loss until a pre-trained target domain model is obtained.

[0071] In combination with the above embodiments, in one implementation, the present invention further provides a sentiment analysis method based on knowledge distillation and passive domain adaptation. In this embodiment, the information maximization loss also includes: sentiment polarity loss.

[0072] The emotion polarity loss of this embodiment is used to calculate the loss of positive emotion categories and negative emotion categories predicted by the target domain model for target domain image data without emotion labels, so as to increase the depth of target domain feature representation. The emotion polarity loss of this embodiment can be calculated based on the loss of category distribution probability and then using polarity probability to calculate the loss, wherein the polarity probability is the sum of the probabilities of the positive category and the sum of the probabilities of the negative category.

[0073] In this embodiment, in order to solve the problem of fuzzy category features of the sentiment dataset, a sentiment polarity loss is proposed to increase the depth of the target domain feature representation and explore the subtle differences of the sentiment dataset. By observing the unique feature distribution patterns of different sentiment categories, it can be found that categories with matching sentiment polarity (positive or negative) are more closely associated. Therefore, this embodiment combines sentiment polarity (i.e., sentiment polarity loss) with self-label loss (i.e., third loss) and information maximization loss ( ) are considered together. The sentiment polarity loss of this embodiment aims to explore multiple dimensions and nuances of emotions, thereby promoting a more complex understanding of emotional states. This embodiment introduces sentiment polarity loss to overcome the ambiguity of sentiment labels.

[0074] In an alternative embodiment, the information maximization loss including the sentiment polarity loss Can be: ;

[0075] in, , . To calculate the expectation; represents the target domain sample, k represents the category label, K represents the label space, represents the probability of predicting category k, e represents the polarity label, pos represents the positive category, neg represents the negative category, Represents the probability of predicting the polarity label e.

[0076] In this embodiment, the target domain model is trained from scratch, and inter-domain alignment is performed between the bridge domain model and the target domain model, which allows the target domain model to learn feature relationships independent of the source domain, thereby expanding the exploration of target-specific features. In addition, in order to solve the problem of intra-domain feature ambiguity, sentiment polarity constraints are introduced to enhance the target model's ability to discriminate sentiment categories.

[0077] In combination with the above embodiments, in one implementation, the present invention further provides a sentiment analysis method based on knowledge distillation and passive domain adaptation. In this method, the target domain image data without emotion labels includes: original target domain image data without emotion labels and original target domain image data without emotion labels after masking; and the "inputting the target domain image data without emotion labels into the bridge domain model to be trained" in the above step S22 may specifically include step S41:

[0078] Step S41: inputting the target domain image data without emotion labels into the bridge domain model to be trained, and based on the emotion prediction results corresponding to the original target domain image data without emotion labels and the original target domain image data without emotion labels after masking, using the exponential moving average strategy EMA to update the model parameters of the feature extractor of the bridge domain model, so that the feature extractor of the bridge domain model learns local and global emotion semantic information.

[0079] In this embodiment, different blocks in an emotional image are usually associated with a unified emotional label, which is different from the standard classification task in which different blocks in an image correspond to different semantics. Taking this into account, this embodiment also introduces a fragment-based masking learning module, that is, a masking enhancement framework is proposed to promote emotional semantic connection, thereby preventing the model from ignoring contextual relationships.

[0080] Specifically, this embodiment performs masking processing on the original emotion-free label target domain image data through a masking enhancement framework to obtain the original emotion-free label target domain image data after masking processing, so that the original emotion-free label target domain image data after masking processing and the original emotion-free label target domain image data are used as training samples of the bridging model for model training.

[0081] Then, the target domain image data without emotion labels (i.e., the original target domain image data without emotion labels and the original target domain image data without emotion labels after masking) are input into the bridge domain model to be trained, and the emotion prediction results corresponding to the original target domain image data without emotion labels and the original target domain image data without emotion labels after masking are obtained. Based on the emotion prediction results corresponding to the original target domain image data without emotion labels and the original target domain image data without emotion labels after masking, the exponential moving average strategy EMA is used to update the model parameters of the feature extractor of the bridge domain model to obtain the trained bridge domain model, so that the feature extractor of the bridge domain model can learn local and global emotional semantic information.

[0082] In this embodiment, a masking enhancement framework is proposed to utilize the sentiment consistency in different contexts. Masking at the patch level will produce a larger receptive field, thereby exploring stronger contextual information and encouraging the model to focus on local features in order to more effectively extract sentiment-consistent features, thereby achieving better classification results. After masking, the feature extractor is forced to learn local and global sentiment semantic information using sparse residual cues.

[0083] In combination with the above embodiments, in one implementation, the present invention further provides a sentiment analysis method based on knowledge distillation and passive domain adaptation. In this method, the above step S21 may specifically include steps S51 to S54:

[0084] Step S51: inputting the target domain image data without emotion labels into the source domain model to obtain a first initial pseudo emotion label of the target domain image data without emotion labels.

[0085] In this embodiment, the target domain image data without emotion labels is input into a pre-trained source domain model to obtain a first initial pseudo emotion label corresponding to the target domain image data without emotion labels.

[0086] Step S52: Calculate the Euclidean distance, cosine distance and Manhattan distance between the features of the target domain image data without emotion labels.

[0087] In this embodiment, based on the hybrid distance measurement method, the distance between the features of the target domain image data without emotion labels is measured. Specifically, the Euclidean distance, cosine distance and Manhattan distance between the features of the target domain image data without emotion labels can be calculated respectively.

[0088] Step S53: performing weighted summation on the Euclidean distance, the cosine distance and the Manhattan distance to obtain a mixed distance between the features of the target domain image data without emotion labels.

[0089] In this embodiment, after obtaining the Euclidean distance, cosine distance and Manhattan distance between the features of the target domain image data without emotion labels, the Euclidean distance, cosine distance and Manhattan distance can be weightedly summed to obtain a mixed distance between the features of the target domain image data without emotion labels.

[0090] Traditional distance measurement methods often use only one distance measurement strategy, such as cosine distance. However, no distance measurement is always better than other measurements, and each setting tends to favor a specific single measurement. This is because different distance metrics focus on different parts: cosine distance cannot provide amplitude information; Euclidean distance is invariant to translation and rotation, but sensitive to the range of input attributes; Manhattan distance remains unchanged with coordinate system reflection, but is sensitive to rotation. Based on this, this embodiment combines these three distance measurement methods to propose a hybrid distance measurement, which improves the robustness in different data types.

[0091] Step S54: Based on the mixed distance between the features of the target domain image data without emotion labels, k-means is used for clustering, the category midpoint of the first initial pseudo emotion label is reallocated, and the first initial pseudo emotion label is corrected to obtain the first pseudo emotion label of the target domain image data without emotion labels.

[0092] In this embodiment, k-means can be used for clustering based on the mixed distance between the features of the target domain image data without emotion labels, and the category midpoint of the first initial pseudo emotion label (i.e., the center point of the category) can be redistributed to correct the first initial pseudo emotion label to obtain the first pseudo emotion label corresponding to the target domain image data without emotion labels.

[0093] In this embodiment, for the training of the bridge domain model, in the absence of target domain labels, the distillation strategy cannot alleviate the disturbance caused by label mismatch. To address this challenge, this embodiment adopts clustering as a label refinement strategy, which imposes additional constraints on the model output. Specifically, weighted mean clustering is used to calculate the center point of each class in the target domain. In order to obtain a better distance metric, a fusion distance metric is further adopted to calculate the Euclidean distance, cosine distance, and Manhattan distance between two features, so as to use the clustering-enhanced source domain model to generate a more accurate first pseudo sentiment label, which is conducive to retaining the unchanged knowledge of the source domain.

[0094] And, in an optional embodiment, in order to reduce erroneous pseudo-labels, pseudo-label post-processing and masking strategies based on hybrid distance metric clustering can be introduced simultaneously to restrict the distribution of pseudo-labels and explore richer semantic contexts.

[0095] In one embodiment, if Figure 3 As shown, Figure 3FIG. 1 is a schematic diagram showing a comparison between a traditional passive domain adaptation method and a passive emotion domain adaptation method based on knowledge distillation according to an embodiment of the present invention. Figure 3 middle, Figure 3 (a) in the figure is a traditional passive domain adaptation method, which is based on the source data and source domain labels to perform supervised training of the source domain model. After the source domain model is trained, the feature extractor of the source domain model is initialized with the feature extractor of the target domain model to adapt from the edge domain to the target domain. Then, the target domain model is trained based on the unlabeled target domain data to obtain the trained target domain model. In other words, it aligns different domains by directly fine-tuning the source domain model on the target domain.

[0096] Figure 3 (b) in the figure is a passive emotion domain adaptation method based on knowledge distillation proposed in an embodiment of the present invention. In this method, the source domain model is trained based on labeled source domain data. , get the trained source domain model The feature extractor and classifier of The losses include at least (i.e. the first loss function).

[0097] In the passive emotion domain adaptation method based on knowledge distillation, considering the ambiguity between domains and within domains in the emotion dataset, as well as the emotion and mood gap, the passive emotion domain adaptation method based on knowledge distillation of this embodiment needs to maintain a balance between retaining the source-invariant domain knowledge and reducing the noise of source-specific knowledge. That is, in order to solve the two problems of the unclear distribution of emotion data features leading to the difficulty of direct fine-tuning, and the low label confidence leading to erroneous pseudo-labels, this embodiment proposes a new passive domain adaptation framework suitable for visual emotion recognition, which mainly includes two steps: SBMG (source-preserving bridging model generation) and ETMA (emotion-aligned target model adaptation).

[0098] For unlabeled target domain data, SBMG generates a bridge domain model To align sentiment data from different domains, the challenges of directly fine-tuning sentiment data are avoided. The model parameters of the feature extractor are related to the bridge domain model The feature extractor is initialized and the source domain model The model parameters of the classifier are used as the bridge domain model The model parameters of the classifier are fixed to obtain the bridge domain model to be trained, and then the model parameters of the feature extractor of the bridge domain model are updated based on the exponential moving average strategy EMA to obtain the trained bridge domain model. The losses include at least (i.e., the first distillation loss function) and (i.e. the second loss function).

[0099] Next, ETMA restarts training for the unlabeled target domain data and performs inter-domain alignment between the bridge model and the target model. The feature extractor is randomly initialized and the source domain model The model parameters of the classifier are used as the target domain model The model parameters of the classifier are fixed to obtain the target domain model to be trained, and the model parameters of the feature extractor of the target domain model are updated to obtain the trained target domain model. The losses include at least (i.e., the second distillation loss function), (i.e. the third loss function) and the information maximization loss function .

[0100] Compared with traditional passive domain adaptation methods, this embodiment is the first work focusing on passive domain adaptation for visual emotion recognition. This embodiment solves the problem that traditional passive domain adaptation methods cannot well solve the inter-domain and intra-domain ambiguity existing in emotion datasets, improves the performance of passive domain adaptation methods in domain adaptation in emotion datasets, and the learned target domain features are more similar to the original structure of the target domain.

[0101] In addition, in a specific implementation case, all methods implemented in the sentiment analysis method of passive domain adaptation based on knowledge distillation use ResNet-101 as the network backbone, which is pre-trained on ImageNet. The input image at the beginning of ResNet-101 processing has a width and height of 224 pixels and 3 color channels (RGB). The network is first processed by a convolution layer consisting of 7 convolution kernels with a stride of 2, and the output feature map has a width and height of 112 and a number of channels of 64. Then, the image is further reduced to a width and height of 56 through a maximum pooling layer consisting of 3 convolution kernels with a stride of 2, and 64 channels are still maintained. The network mainly performs deep feature learning through four stages of residual blocks, and finally completes the output of the classification task through global average pooling and a fully connected layer.

[0102] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0103] Based on the same inventive concept, an embodiment of the present invention provides a sentiment analysis device based on knowledge distillation and passive domain adaptation. Figure 4 , Figure 4 1 is a structural block diagram of a sentiment analysis device based on knowledge distillation and passive domain adaptation provided by an embodiment of the present invention. Figure 4 As shown, the sentiment analysis device 400 based on knowledge distillation and passive domain adaptation of this embodiment may include:

[0104] A first model acquisition module 401 is used to obtain a pre-trained source domain model, where the source domain model is trained using source domain image data with emotion labels, and the source domain model includes a feature extractor and a classifier;

[0105] The second model acquisition module 402 is used to initialize the feature extractor of the bridge domain model using the model parameters of the feature extractor of the source domain model, and use the model parameters of the classifier of the source domain model as the model parameters of the classifier of the bridge domain model and fix them to obtain the bridge domain model to be trained;

[0106] The first model training module 403 is used to perform knowledge distillation learning using the bridge domain model to be trained as a student model and the pre-trained source domain model as a teacher model to obtain a trained bridge domain model;

[0107] The third model acquisition module 404 is used to randomly initialize the model parameters of the feature extractor of the target domain model, and use the model parameters of the classifier of the pre-trained source domain model as the model parameters of the classifier of the target domain model and fix them to obtain the target domain model to be trained;

[0108] The second model training module 405 is used to perform knowledge distillation learning with the target domain model to be trained as a student model and the trained bridge domain model as a teacher model to obtain a trained target domain model, wherein the loss function for training the target domain model at least includes an alignment loss function, and the alignment loss function is used to align the sentiment predictions of the bridge domain model and the target domain model;

[0109] The sentiment analysis module 406 is used to use the trained target domain model to perform sentiment classification prediction on the image data of the target domain to obtain a sentiment analysis result.

[0110] Optionally, the first model training module 403 includes:

[0111] A first label determination module, used for inputting the target domain image data without emotion label into the source domain model to obtain a first pseudo emotion label of the target domain image data without emotion label;

[0112] The bridge domain model training module is used to input the target domain image data without emotion labels into the bridge domain model to be trained, and based on the first pseudo emotion label of the target domain image data without emotion labels, use the exponential moving average strategy EMA to update the model parameters of the feature extractor of the bridge domain model to obtain the trained bridge domain model.

[0113] Optionally, the loss function for training the source domain model includes at least a first loss function, and the first loss function is used to calculate the loss between the emotion prediction result obtained by the source domain model based on the source domain image data with emotion labels and the emotion label corresponding to the source domain image data with emotion labels;

[0114] The loss function for training the bridge domain model includes at least: a second loss function and a first distillation loss function;

[0115] The second loss function is used to calculate the loss between the emotion prediction result obtained by the bridge domain model based on the target domain image data without emotion label and the first pseudo emotion label of the target domain image data without emotion label;

[0116] The first distillation loss function is used to calculate the KL divergence loss between the distribution of the emotion prediction results obtained by the bridge domain model based on the target domain image data without emotion labels and the distribution of the emotion prediction results obtained by the source domain model based on the target domain image data without emotion labels.

[0117] Optionally, the second model training module 405 includes:

[0118] A second label determination module is used to input the target domain image data without emotion labels into the trained bridge domain model to obtain a second pseudo emotion label for the target domain image data without emotion labels;

[0119] The target domain model training module is used to input the target domain image data without emotion labels into the target domain model to be trained, and based on the second pseudo emotion label of the target domain image data without emotion labels, update the model parameters of the feature extractor of the target domain model to obtain the trained target domain model.

[0120] Optionally, the loss function for training the target domain model further includes: information maximization loss; the alignment loss function at least includes: a third loss function and a second distillation loss function;

[0121] The third loss function is used to calculate the loss between the emotion prediction result obtained by the target domain model based on the target domain image data without emotion label and the second pseudo emotion label of the target domain image data without emotion label;

[0122] The second distillation loss function is used to calculate the KL divergence loss between the distribution of the emotion prediction results obtained by the bridge domain model based on the target domain image data without emotion labels and the distribution of the emotion prediction results obtained by the target domain model based on the target domain image data without emotion labels;

[0123] The information maximization loss is used to calculate the prediction confidence maximization loss of each emotion category predicted by the target domain model for the target domain image data without emotion labels.

[0124] Optionally, the information maximization loss further includes: sentiment polarity loss;

[0125] The sentiment polarity loss is used to calculate the loss of positive sentiment categories and negative sentiment categories predicted by the target domain model for the target domain image data without sentiment labels, so as to increase the depth of target domain feature representation.

[0126] Optionally, the target domain image data without emotion labels includes: original target domain image data without emotion labels and original target domain image data without emotion labels after masking; the bridge domain model training module includes:

[0127] The masked model training module is used to input the target domain image data without emotion labels into the bridge domain model to be trained, so as to update the model parameters of the feature extractor of the bridge domain model using the exponential moving average strategy EMA based on the emotion prediction results corresponding to the original target domain image data without emotion labels and the original target domain image data without emotion labels after masking, so that the feature extractor of the bridge domain model can learn local and global emotion semantic information.

[0128] Optionally, the first tag determination module includes:

[0129] An initial pseudo-label determination module, used for inputting the target domain image data without emotion labels into the source domain model to obtain a first initial pseudo-emotion label for the target domain image data without emotion labels;

[0130] A single distance calculation module, used to calculate the Euclidean distance, cosine distance and Manhattan distance between the features of the target domain image data without emotion labels;

[0131] A mixed distance calculation module, used for performing weighted summation on the Euclidean distance, the cosine distance and the Manhattan distance to obtain a mixed distance between features of the target domain image data without emotion labels;

[0132] A pseudo-label correction module is used to perform clustering using k-means based on the mixed distance between the features of the target domain image data without emotion labels, reallocate the category midpoint of the first initial pseudo-emotion label, correct the first initial pseudo-emotion label, and obtain the first pseudo-emotion label of the target domain image data without emotion labels.

[0133] Based on the same inventive concept, another embodiment of the present invention provides an electronic device 500, such as Figure 5 shown. Figure 5 is a schematic diagram of an electronic device shown in an embodiment of the present invention. The electronic device includes a memory 502, a processor 501, and a computer program stored in the memory and executable on the processor, and when the processor executes, the steps in the sentiment analysis method based on knowledge distillation and passive domain adaptation described in any of the above embodiments of the present invention are implemented.

[0134] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0135] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0136] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, embodiments of the present invention may take the form of complete hardware embodiments, complete software embodiments, or embodiments combining software and hardware. Furthermore, embodiments of the present invention may take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0137] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0138] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0140] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0141] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0142] The above is a detailed introduction to the sentiment analysis method, device and equipment based on knowledge distillation and passive domain adaptation provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A sentiment analysis method based on knowledge distillation and passive domain adaptation, characterized in that: The method comprises: Obtaining a pre-trained source domain model, where the source domain model is trained using source domain image data with emotion labels, and the source domain model includes a feature extractor and a classifier; Initializing the feature extractor of the bridge domain model using the model parameters of the feature extractor of the source domain model, taking the model parameters of the classifier of the source domain model as the model parameters of the classifier of the bridge domain model and fixing them, thereby obtaining the bridge domain model to be trained; Using the bridge domain model to be trained as a student model and the pre-trained source domain model as a teacher model, knowledge distillation learning is performed to obtain a trained bridge domain model; The model parameters of the feature extractor of the target domain model are randomly initialized, and the model parameters of the classifier of the pre-trained source domain model are used as the model parameters of the classifier of the target domain model and fixed to obtain the target domain model to be trained; The target domain model to be trained is used as a student model, and the trained bridge domain model is used as a teacher model to perform knowledge distillation learning to obtain a trained target domain model, wherein the loss function for training the target domain model at least includes an alignment loss function, and the alignment loss function is used to align the sentiment predictions of the bridge domain model and the target domain model; Using the trained target domain model, sentiment classification prediction is performed on the image data of the target domain to obtain a sentiment analysis result.

2. The sentiment analysis method based on passive domain adaptation based on knowledge distillation according to claim 1, characterized in that: The method uses the bridge domain model to be trained as a student model and the pre-trained source domain model as a teacher model to perform knowledge distillation learning to obtain a trained bridge domain model, including: Inputting the target domain image data without emotion labels into the source domain model to obtain a first pseudo emotion label for the target domain image data without emotion labels; The target domain image data without emotion labels is input into the bridge domain model to be trained, and based on the first pseudo emotion label of the target domain image data without emotion labels, the model parameters of the feature extractor of the bridge domain model are updated by using the exponential moving average strategy EMA to obtain the trained bridge domain model.

3. The sentiment analysis method based on passive domain adaptation based on knowledge distillation according to claim 2, characterized in that: The loss function for training the source domain model includes at least a first loss function, and the first loss function is used to calculate the loss between the emotion prediction result obtained by the source domain model based on the source domain image data with emotion labels and the emotion label corresponding to the source domain image data with emotion labels; The loss function for training the bridge domain model includes at least: a second loss function and a first distillation loss function; The second loss function is used to calculate the loss between the emotion prediction result obtained by the bridge domain model based on the target domain image data without emotion label and the first pseudo emotion label of the target domain image data without emotion label; The first distillation loss function is used to calculate the KL divergence loss between the distribution of the emotion prediction results obtained by the bridge domain model based on the target domain image data without emotion labels and the distribution of the emotion prediction results obtained by the source domain model based on the target domain image data without emotion labels.

4. The sentiment analysis method based on passive domain adaptation based on knowledge distillation according to claim 1, characterized in that: The method uses the target domain model to be trained as a student model and the trained bridge domain model as a teacher model to perform knowledge distillation learning to obtain a trained target domain model, including: Inputting the target domain image data without emotion labels into the trained bridge domain model to obtain a second pseudo emotion label for the target domain image data without emotion labels; The target domain image data without emotion labels is input into the target domain model to be trained, and based on the second pseudo emotion label of the target domain image data without emotion labels, the model parameters of the feature extractor of the target domain model are updated to obtain the trained target domain model.

5. The sentiment analysis method based on passive domain adaptation based on knowledge distillation according to claim 4, characterized in that: The loss function for training the target domain model also includes: information maximization loss; the alignment loss function at least includes: a third loss function and a second distillation loss function; The third loss function is used to calculate the loss between the emotion prediction result obtained by the target domain model based on the target domain image data without emotion label and the second pseudo emotion label of the target domain image data without emotion label; The second distillation loss function is used to calculate the KL divergence loss between the distribution of the emotion prediction results obtained by the bridge domain model based on the target domain image data without emotion labels and the distribution of the emotion prediction results obtained by the target domain model based on the target domain image data without emotion labels; The information maximization loss is used to calculate the prediction confidence maximization loss of each emotion category predicted by the target domain model for the target domain image data without emotion labels.

6. The sentiment analysis method based on passive domain adaptation based on knowledge distillation according to claim 5, characterized in that: The information maximization loss also includes: sentiment polarity loss; The sentiment polarity loss is used to calculate the loss of positive sentiment categories and negative sentiment categories predicted by the target domain model for the target domain image data without sentiment labels, so as to increase the depth of target domain feature representation.

7. The sentiment analysis method based on passive domain adaptation based on knowledge distillation according to claim 2 or 3, characterized in that: The target domain image data without emotion labels includes: original target domain image data without emotion labels and original target domain image data without emotion labels after masking; the step of inputting the target domain image data without emotion labels into the bridge domain model to be trained includes: The target domain image data without emotion labels is input into the bridge domain model to be trained, and based on the emotion prediction results corresponding to the original target domain image data without emotion labels and the masked original target domain image data without emotion labels, the exponential moving average strategy EMA is used to update the model parameters of the feature extractor of the bridge domain model, so that the feature extractor of the bridge domain model learns local and global emotion semantic information.

8. The sentiment analysis method based on passive domain adaptation based on knowledge distillation according to claim 2 or 3, characterized in that: The step of inputting the target domain image data without emotion labels into the source domain model to obtain a first pseudo emotion label for the target domain image data without emotion labels comprises: Inputting the target domain image data without emotion labels into the source domain model to obtain a first initial pseudo emotion label for the target domain image data without emotion labels; Calculating the Euclidean distance, cosine distance and Manhattan distance between the features of the target domain image data without emotion labels; Performing a weighted summation on the Euclidean distance, the cosine distance, and the Manhattan distance to obtain a mixed distance between features of the target domain image data without emotion labels; Based on the mixed distance between the features of the target domain image data without emotion labels, k-means is used for clustering, the category midpoint of the first initial pseudo emotion label is redistributed, the first initial pseudo emotion label is corrected, and the first pseudo emotion label of the target domain image data without emotion labels is obtained.

9. A sentiment analysis device based on knowledge distillation and passive domain adaptation, characterized in that: The device comprises: A first model acquisition module is used to obtain a pre-trained source domain model, where the source domain model is trained using source domain image data with emotion labels, and the source domain model includes a feature extractor and a classifier; A second model acquisition module is used to initialize the feature extractor of the bridge domain model using the model parameters of the feature extractor of the source domain model, and to fix the model parameters of the classifier of the source domain model as the model parameters of the classifier of the bridge domain model, so as to obtain the bridge domain model to be trained; A first model training module is used to perform knowledge distillation learning using the bridge domain model to be trained as a student model and the pre-trained source domain model as a teacher model to obtain a trained bridge domain model; A third model acquisition module is used to randomly initialize the model parameters of the feature extractor of the target domain model, and use the model parameters of the classifier of the pre-trained source domain model as the model parameters of the classifier of the target domain model and fix them to obtain the target domain model to be trained; A second model training module is used to perform knowledge distillation learning using the target domain model to be trained as a student model and the trained bridge domain model as a teacher model to obtain a trained target domain model, wherein the loss function for training the target domain model at least includes an alignment loss function, and the alignment loss function is used to align the sentiment predictions of the bridge domain model and the target domain model; The sentiment analysis module is used to use the trained target domain model to perform sentiment classification prediction on the image data of the target domain to obtain a sentiment analysis result.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by the processor, the sentiment analysis method based on passive domain adaptation based on knowledge distillation is implemented as described in any one of claims 1 to 8.

Images