Canine emotion recognition method based on cross-species feature learning ability transfer

Abstract

This disclosure provides a canine emotion recognition method based on cross-species feature learning ability transfer, belonging to the field of data recognition technology. Specifically, it includes: constructing a human feature learning model using a 3D convolutional neural network and a visual transformer network based on a two-stage feature learning paradigm; performing advanced multi-task collaborative pre-training on the human feature learning model; simplifying and compressing the 3D convolutional neural network and the visual transformer network to construct a canine feature learning model; designing a cross-domain knowledge composite distillation technique to transfer human facial expression feature learning ability to the canine feature learning model; connecting the canine feature learning model with a softmax classifier to form a canine emotion recognition framework; training the framework using a canine training set; and using the trained canine emotion recognition framework to recognize target canine video data. This disclosure improves the adaptability and accuracy of emotion recognition.

Description

Technical Field

[0001] This disclosure relates to the field of data recognition technology, and in particular to a method for canine emotion recognition based on cross-species feature learning ability transfer. Background Technology

[0002] Canine emotion recognition technology aims to identify potential emotions such as happiness, sadness, and anger by learning canine facial expressions. This technology can promptly capture dangerous emotions in dogs, predict dangerous behaviors, and reduce the threat posed by dogs to humans.

[0003] Currently, the recognition of canine emotions primarily relies on video data of dogs collected via cameras and their associated emotion labels to construct a training set. This training set is used to train an emotion recognition framework based on deep learning algorithms, enabling the framework to learn facial expression features (such as facial texture and facial features) and postural features (such as body language and movements) for emotion recognition. Considering the scarcity and difficulty in acquiring canine data, current research utilizes large-scale image datasets representing thousands of animal categories to pre-train the feature learning model within the emotion recognition framework, enabling it to extract general visual features. Then, canine data is used to train and fine-tune the emotion recognition framework.

[0004] However, the current method has two problems. First, the pre-training process focuses on animal category discrimination, causing the feature learning model to concentrate on easily distinguishable morphological features such as skeletal structure and body shape, neglecting the extraction of facial expression and posture features, thus limiting the visual feature learning ability of the feature learning model. Second, the fine-tuning process directly applies the feature learning model to canine emotion recognition without considering the model's task adaptability to canine emotion recognition, affecting recognition performance.

[0005] It is evident that there is an urgent need for a canine emotion recognition method based on cross-species feature learning transfer with high adaptability and recognition accuracy. Summary of the Invention

[0006] In view of this, the present disclosure provides a canine emotion recognition method based on cross-species feature learning ability transfer, which at least partially solves the problems of poor adaptability and recognition accuracy in the prior art.

[0007] This disclosure provides a canine emotion recognition method based on cross-species feature learning ability transfer, including:

[0008] Step 1: Based on the two-stage feature learning paradigm, construct a human feature learning model using 3D convolutional neural networks and visual transformer networks;

[0009] Step 2: Design pre-tasks related to human facial expressions to conduct advanced multi-task collaborative pre-training of the human feature learning model.

[0010] Step 3: Simplify and compress the 3D convolutional neural network and the visual transformer network to construct a canine feature learning model;

[0011] Step 4: Design a cross-domain knowledge composite distillation technique to transfer the human facial expression feature learning ability to a canine feature learning model;

[0012] Step 5: Connect the canine feature learning model with the softmax classifier to form a canine emotion recognition framework. Train the canine emotion recognition framework using the canine training set, and use the trained canine emotion recognition framework to recognize target canine video data.

[0013] According to one specific implementation of this disclosure, the visual transformer network includes N transformer modules, where N is an integer greater than 2.

[0014] According to a specific implementation of this disclosure, the calculation process of any transformer module in the human feature learning model is as follows:

[0015]

[0016]

[0017]

[0018] Among them, z n-1 This represents the human representation features obtained by the (n-1)th Transformer module, and STA(*) represents the overall attention mechanism. This represents the attention features obtained from the first overall attention mechanism, and LN(*) represents the normalization operation. This represents the attention features obtained from the second global attention mechanism, DS(*) represents the dense mapping, and z n This represents the nth human representational feature obtained at the end.

[0019] According to a specific implementation of this disclosure, the human expression-related pre-task includes a context learning pre-task, a facial expression learning pre-task, and a posture expression learning pre-task.

[0020] The context learning pre-tasks include reverse chronological prediction, frame skipping estimation, and intermediate frame semantic segmentation.

[0021] The steps of the reverse chronological prediction specifically include performing reverse chronological prediction on the original human behavior video frame sequence;

[0022] The frame skipping estimation steps specifically include randomly skipping a frame to obtain the remaining frames and forming two sub-video segments, and using these two sub-video segments to estimate the content of the skipped frame;

[0023] The intermediate frame semantic segmentation steps specifically include semantic segmentation of human structures in intermediate video frames.

[0024] The steps of the pre-task for facial expression learning specifically include facial disorder reconstruction and facial defect estimation.

[0025] The steps of the pre-task for learning posture and expression specifically include gesture skeleton estimation, limb skeleton angle calculation, and key part capture.

[0026] According to a specific implementation of an embodiment of this disclosure, step 2 specifically includes:

[0027] The first stage of pre-training is performed on the 3D convolutional neural network in the human feature learning model using a context learning pre-task.

[0028] By connecting a 3D convolutional neural network with a visual transformer network, and performing pre-training tasks for facial expression learning and pose expression learning, the connected human feature learning model is trained in the second stage of collaborative pre-training, thus completing the pre-training process.

[0029] According to a specific implementation of this disclosure, the calculation process of any transformer module in the canine feature learning model is as follows:

[0030]

[0031]

[0032] Where, x n-1 This represents the canine representation features obtained by the (n-1)th Transformer module. x represents the attentional features obtained from the overall attention mechanism. n This represents the nth canine appearance feature obtained at the end.

[0033] According to a specific implementation of an embodiment of this disclosure, the cross-domain knowledge composite distillation technology includes context feature distillation and facial expression output distillation;

[0034] Step 4 specifically includes:

[0035] Let the parameters of the first m clusters of convolutional layers in the 3D convolutional neural network of the human feature learning model be... The parameters of the first m clusters of convolutional layers in the canine feature learning model are used as The mean squared error of the outputs of the two types of models is calculated as the context feature distillation loss;

[0036] Minimizing the KL divergence between the feature learning model and the target label transfers the ability to learn human facial expression features to the canine feature learning model.

[0037] According to a specific implementation of this disclosure, the formula for calculating the context feature distillation loss is as follows:

[0038]

[0039] Among them, V h and V d These are videos of human scenes and videos of canine scenes. and These represent the first m clusters of 3D convolution operations in the human feature learning model and the canine feature learning model, respectively. This represents the context feature distillation loss corresponding to the first m clusters of convolutional layers.

[0040] According to a specific implementation of this disclosure, the expression for the KL divergence is:

[0041]

[0042] Among them, L T This indicates a loss in facial expression output. and These represent whether the human-themed video scene belongs to or does not belong to the t-th video segment. h The probability of an individual's facial expression. and These respectively indicate whether the dog-related scene video belongs to or does not belong to the t-th video. d The probability of a canine facial expression.

[0043] According to a specific implementation of this disclosure, before step 5, the method further includes:

[0044] The fusion distillation loss is calculated based on the context feature distillation loss and the facial expression output loss.

[0045]

[0046] Where, α f and α T These represent the weights of the context feature distillation loss and the expression output loss, respectively.

[0047] The canine emotion recognition scheme based on cross-species feature learning ability transfer in this embodiment includes: Step 1, constructing a human feature learning model using a 3D convolutional neural network and a visual transformer network based on a two-stage feature learning paradigm; Step 2, designing human expression-related pre-tasks to perform advanced multi-task collaborative pre-training on the human feature learning model; Step 3, simplifying and compressing the 3D convolutional neural network and the visual transformer network to construct a canine feature learning model; Step 4, designing a cross-domain knowledge composite distillation technique to transfer human expression feature learning ability to the canine feature learning model; Step 5, connecting the canine feature learning model with a softmax classifier to form a canine emotion recognition framework, training the canine emotion recognition framework using a canine training set, and using the trained canine emotion recognition framework to recognize target canine video data.

[0048] The beneficial effects of this disclosure are as follows: A human feature learning model is constructed using the scheme disclosed herein, and a multi-attribute pre-training task related to human facial expressions is designed to perform advanced multi-task collaborative pre-training on this model. Then, the human feature learning model is simplified and compressed to construct a canine feature learning model. Furthermore, the human feature learning capability is transferred to the canine feature learning model using cross-domain knowledge distillation technology. Finally, the canine feature learning model is fine-tuned using a canine training set for canine emotion recognition, thereby improving the adaptability and accuracy of emotion recognition. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating a canine emotion recognition method based on cross-species feature learning ability transfer, as provided in an embodiment of this disclosure. Detailed Implementation

[0051] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0052] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0053] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0054] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0055] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0056] This disclosure provides a canine emotion recognition method based on cross-species feature learning ability transfer, which can be applied to image data analysis.

[0057] See Figure 1 This is a flowchart illustrating a canine emotion recognition method based on cross-species feature learning transfer, as provided in an embodiment of this disclosure. Figure 1 As shown, the method mainly includes the following steps:

[0058] Step 1: Based on the two-stage feature learning paradigm, construct a human feature learning model using 3D convolutional neural networks and visual transformer networks;

[0059] Furthermore, the visual transformer network includes N transformer modules, where N is an integer greater than 2.

[0060] Furthermore, the computation process of any transformer module in the human feature learning model is as follows:

[0061]

[0062]

[0063]

[0064] Among them, z n-1 This represents the human representation features obtained by the (n-1)th Transformer module, and STA(*) represents the overall attention mechanism. This represents the attention features obtained from the first overall attention mechanism, and LN(*) represents the normalization operation. This represents the attention features obtained from the second global attention mechanism, DS(*) represents the dense mapping, and z n This represents the nth human representational feature obtained at the end.

[0065] In practical implementation, considering the complexity and diversity of human expressions, a human feature learning model is constructed based on a two-stage feature learning paradigm. This model consists of two parts: a 3D convolutional neural network and a visual transformer network. The 3D convolutional neural network is composed of M clusters of convolutional layers at different scales, used to extract deep convolutional features describing environmental context information. The visual transformer network consists of N transformer modules, which learn the overall features of human behavior in the deep convolutional features through a holistic attention mechanism. After normalizing these features, a dense mapping is performed to obtain human representation features. The calculation process of the nth visual transformer module is shown in equations (1) to (3):

[0066]

[0067]

[0068]

[0069] In equation (1), z n-1 This represents the human representation features obtained by the (n-1)th visual Transformer, and STA(*) represents the overall attention mechanism. This represents the attentional features obtained from the first overall attention mechanism. In equation (2), LN(*) represents the normalization operation. This represents the attentional features obtained from the second overall attention mechanism. In equation (3), DS(*) represents the dense mapping, z n This represents the nth human representational feature obtained at the end.

[0070] Step 2: Design pre-tasks related to human facial expressions to conduct advanced multi-task collaborative pre-training of the human feature learning model.

[0071] Furthermore, the human expression-related pre-tasks include context learning pre-tasks, facial expression learning pre-tasks, and posture expression learning pre-tasks.

[0072] The context learning pre-tasks include reverse chronological prediction, frame skipping estimation, and intermediate frame semantic segmentation.

[0073] The steps of the reverse chronological prediction specifically include performing reverse chronological prediction on the original human behavior video frame sequence;

[0074] The frame skipping estimation steps specifically include randomly skipping a frame to obtain the remaining frames and forming two sub-video segments, and using these two sub-video segments to estimate the content of the skipped frame;

[0075] The intermediate frame semantic segmentation steps specifically include semantic segmentation of human structures in intermediate video frames.

[0076] The steps of the pre-task for facial expression learning specifically include facial disorder reconstruction and facial defect estimation.

[0077] The steps of the pre-task for learning posture and expression specifically include gesture skeleton estimation, limb skeleton angle calculation, and key part capture.

[0078] Furthermore, step 2 specifically includes:

[0079] The first stage of pre-training is performed on the 3D convolutional neural network in the human feature learning model using a context learning pre-task.

[0080] By connecting a 3D convolutional neural network with a visual transformer network, and performing pre-training tasks for facial expression learning and pose expression learning, the connected human feature learning model is trained in the second stage of collaborative pre-training, thus completing the pre-training process.

[0081] In practical implementation, the pre-training tasks are designed from global to local levels, including context learning, facial expression learning, and pose expression learning. The context learning tasks are reverse chronological prediction, frame skipping estimation, and intermediate frame semantic segmentation. Reverse chronological prediction involves predicting the original sequence of human behavior video frames in reverse order. Frame skipping estimation involves randomly skipping a frame to obtain two sub-video segments, and then using these two sub-video segments to estimate the content of the skipped frame. Intermediate frame semantic segmentation involves semantically segmenting the human structure in the intermediate frames of the video. The context learning tasks are used to learn the overall semantic features of the human behavior context, providing basic learning capabilities for subsequent learning of detailed features related to human expressions. The facial expression learning tasks are facial disorder reconstruction and facial incompleteness estimation. Disorder reconstruction aims to reconstruct the originally correctly arranged facial image using disordered facial image patches. Incompleteness estimation uses the remaining facial image patches to estimate the missing parts. The facial expression learning tasks are used to learn human facial expression features. The pre-tasks for posture and facial expression learning include hand gesture skeleton estimation, limb skeleton angle calculation, and key feature capture. Hand gesture skeleton estimation refers to estimating the 3D positions of various joints in the human hand. Limb skeleton angle calculation involves calculating the angles between different positions of the human limbs. Key feature capture captures the most crucial parts that describe human posture and facial expressions. The pre-task for facial expression learning is used to learn human posture and facial expression features.

[0082] After completing the pre-training task design, a pre-training process was performed based on a large-scale, broad-based unlabeled human behavior video dataset. First, a context learning pre-task was used to perform the first stage of pre-training on the 3D convolutional neural network in the human feature learning model, enabling it to learn basic semantic information about human behavior, ensuring the subsequent learning of human expressions. Then, the 3D convolutional neural network was connected to a visual transformer network, and pre-tasks for facial expression learning and pose expression learning were used to perform the second stage of collaborative pre-training on the connected human feature learning model, completing the learning of human facial expressions and pose expressions.

[0083] Step 3: Simplify and compress the 3D convolutional neural network and the visual transformer network to construct a canine feature learning model;

[0084] Based on the above embodiments, the calculation process of any transformer module in the canine feature learning model is as follows:

[0085]

[0086]

[0087] Where, x n-1This represents the canine representation features obtained by the (n-1)th Transformer module. x represents the attentional features obtained from the overall attention mechanism. n This represents the nth canine appearance feature obtained at the end.

[0088] In practice, compared to the complex and diverse human expressions, canine expressions are simple and have a single category. Therefore, the human feature learning model cannot be directly used for canine expression learning. Thus, it is necessary to simplify and compress the model while retaining its core learning capabilities. Based on this, a canine feature learning model is constructed using a 3D convolutional neural network and a visual Transformer network. The 3D convolutional neural network is also composed of M clusters of convolutional layers at different scales. The difference is that the number of convolutional layers in each cluster is reduced, thereby weakening the convolutional neural network's ability to learn the overall semantic features of the context. The visual Transformer network is also composed of N visual Transformers. It learns the overall features of canine behavior in the deep convolutional features through the overall attention mechanism. After normalizing the features, dense mapping is performed to obtain the canine behavior features. The calculation process of the nth visual transformer module is shown in equations (4) and (5):

[0089]

[0090]

[0091] In equation (4), x n-1 This represents the canine representation features obtained by the (n-1)th visual Transformer. This represents the attentional characteristics obtained from the overall attention mechanism. In equation (5), x n This represents the nth canine appearance feature obtained at the end.

[0092] Step 4: Design a cross-domain knowledge composite distillation technique to transfer the human facial expression feature learning ability to a canine feature learning model;

[0093] Furthermore, the cross-domain knowledge composite distillation technique includes contextual feature distillation and facial expression output distillation;

[0094] Step 4 specifically includes:

[0095] Let the parameters of the first m clusters of convolutional layers in the 3D convolutional neural network of the human feature learning model be... The parameters of the first m clusters of convolutional layers in the canine feature learning model are used as The mean squared error of the outputs of the two types of models is calculated as the context feature distillation loss;

[0096] Minimizing the KL divergence between the feature learning model and the target label transfers the ability to learn human facial expression features to the canine feature learning model.

[0097] Furthermore, the formula for calculating the context feature distillation loss is as follows:

[0098]

[0099] Among them, V h and V d These are videos of human scenes and videos of canine scenes. and These represent the first m clusters of 3D convolution operations in the human feature learning model and the canine feature learning model, respectively. This represents the context feature distillation loss corresponding to the first m clusters of convolutional layers.

[0100] Furthermore, the expression for the KL divergence is as follows:

[0101]

[0102] Among them, L T This indicates a loss in facial expression output. and These represent whether the human-themed video scene belongs to or does not belong to the t-th video segment. h The probability of an individual's facial expression. and These respectively indicate whether the dog-related scene video belongs to or does not belong to the t-th video. d The probability of a canine facial expression.

[0103] In practical implementation, a cross-domain knowledge distillation technique is designed to transfer the human facial expression feature learning ability to the canine feature learning model. This technique includes contextual feature distillation and facial expression output distillation. Contextual feature distillation forces the convolutional feature learning layers in the canine feature learning model to approximate the responses of the corresponding convolutional feature learning layers in the canine feature learning model, thereby enabling the canine feature learning model to extract overall contextual features. Let the parameters of the first m clusters of convolutional layers in the 3D convolutional neural network of the human feature learning model be denoted as... The parameters of the first m clusters of convolutional layers in the canine feature learning model are used as The mean squared error of the outputs of the two models is calculated as the context feature distillation loss, and the calculation method is as follows:

[0104]

[0105] Where V h and V d These are videos of human scenes and videos of canine scenes. and These represent the first m clusters of 3D convolution operations in the human feature learning model and the canine feature learning model, respectively. This represents the context feature distillation loss corresponding to the first m clusters of convolutional layers. Expression output distillation achieves the transfer of expression feature learning ability by minimizing the KL divergence between the feature learning model and the target label. The KL divergence calculation formula is as follows:

[0106]

[0107] Where L T This indicates a loss in facial expression output. and These represent whether the human-themed video scene belongs to or does not belong to the t-th video segment. h The probability of an individual's facial expression. and These respectively indicate whether the dog-related scene video belongs to or does not belong to the t-th video. d The probability of a canine facial expression.

[0108] Step 5: Connect the canine feature learning model with the softmax classifier to form a canine emotion recognition framework. Train the canine emotion recognition framework using the canine training set, and use the trained canine emotion recognition framework to recognize target canine video data.

[0109] Furthermore, prior to step 5, the method further includes:

[0110] The fusion distillation loss is calculated based on the context feature distillation loss and the facial expression output loss.

[0111]

[0112] Where, α f and α T These represent the weights of the context feature distillation loss and the expression output loss, respectively.

[0113] In practice, we obtain and L T Then, the fusion distillation loss is calculated and used for training the canine feature learning model. The calculation formula is as follows:

[0114]

[0115] Where, α f and α T These represent the weights of the context feature distillation loss and the expression output loss, respectively.

[0116] A canine feature learning model is connected to a softmax classifier to construct a canine emotion recognition framework. A training set is built by collecting canine video data and its emotion labels. The trainable parameters in the canine emotion recognition framework are updated using the backpropagation algorithm to train the model. The trained canine emotion recognition framework is then used for canine emotion recognition.

[0117] This embodiment provides a canine emotion recognition method based on cross-species feature learning ability transfer. It constructs a human feature learning model and designs a multi-attribute pre-training task related to human facial expressions to perform advanced multi-task collaborative pre-training on this model. Then, the human feature learning model is simplified and compressed to construct a canine feature learning model. The method pioneers the concept of cross-species feature learning ability transfer, transferring human facial expression feature learning ability to the canine feature learning model, enabling the model to have efficient facial expression feature extraction capabilities. Furthermore, cross-domain knowledge compound distillation technology is used to transfer human feature learning ability to the canine feature learning model. Finally, the canine feature learning model is fine-tuned using a canine training set for canine emotion recognition, improving the adaptability and accuracy of emotion recognition.

[0118] The units described in the embodiments of this disclosure can be implemented in software or in hardware.

[0119] It should be understood that the various parts of this disclosure can be implemented in hardware, software, firmware, or a combination thereof.

[0120] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for canine emotion recognition based on cross-species feature learning ability transfer, characterized in that, include: Step 1: Based on the two-stage feature learning paradigm, construct a human feature learning model using 3D convolutional neural networks and visual transformer networks; Step 2: Design human expression-related pre-tasks to perform advanced multi-task collaborative pre-training on the human feature learning model. The human expression-related pre-tasks include context learning pre-tasks, facial expression learning pre-tasks, and posture expression learning pre-tasks. The context learning pre-tasks include reverse chronological prediction, frame skipping estimation, and intermediate frame semantic segmentation. The steps of the reverse chronological prediction specifically include performing reverse chronological prediction on the original human behavior video frame sequence; The frame skipping estimation steps specifically include randomly skipping a frame to obtain the remaining frames and forming two sub-video segments, and using these two sub-video segments to estimate the content of the skipped frame; The intermediate frame semantic segmentation steps specifically include semantic segmentation of human structures in intermediate video frames. The steps of the pre-task for facial expression learning specifically include facial disorder reconstruction and facial defect estimation. The steps of the pre-task for learning posture and expression specifically include gesture skeleton estimation, limb skeleton angle calculation, and key part capture. Step 2 specifically includes: The first stage of pre-training is performed on the 3D convolutional neural network in the human feature learning model using a context learning pre-task. The 3D convolutional neural network is connected to the visual transformer network. The pre-tasks of facial expression learning and pose expression learning are used to perform the second stage of collaborative pre-training on the connected human feature learning model, thus completing the pre-training. Step 3: Simplify and compress the 3D convolutional neural network and the visual transformer network to construct a canine feature learning model; Step 4: Design a cross-domain knowledge composite distillation technique to transfer human facial expression feature learning ability to a canine feature learning model. The cross-domain knowledge composite distillation technique includes context feature distillation and facial expression output distillation. Step 4 specifically includes: Let the parameters of the first m clusters of convolutional layers in the 3D convolutional neural network of the human feature learning model be... The parameters of the first m clusters of convolutional layers in the canine feature learning model are used as... The mean squared error of the outputs of the two models is calculated as the context feature distillation loss, wherein the formula for calculating the context feature distillation loss is as follows: in, and These are videos of human scenes and videos of canine scenes. and These represent the first m clusters of 3D convolution operations in the human feature learning model and the canine feature learning model, respectively. This represents the context feature distillation loss corresponding to the first m clusters of convolutional layers; Minimizing the KL divergence between the feature learning model and the target label transfers the ability to learn human facial expression features to the canine feature learning model. The expression for the KL divergence is: in, This indicates a loss in facial expression output. and These respectively indicate whether the human-themed video belongs to or does not belong to the first category. The probability of an individual's facial expression. and These respectively indicate whether the dog-related scene videos belong to and do not belong to the category. The probability of a canine facial expression; Step 5: Connect the canine feature learning model with the softmax classifier to form a canine emotion recognition framework. Train the canine emotion recognition framework using the canine training set, and use the trained canine emotion recognition framework to recognize target canine video data. Before step 5, the method further includes: The fusion distillation loss is calculated based on the context feature distillation loss and the facial expression output loss. in, and These represent the weights of the context feature distillation loss and the facial expression output loss, respectively. express Clusters of convolutional layers at different scales.

2. The method according to claim 1, characterized in that... The visual transformer network comprises N transformer modules, where N is an integer greater than 2.

3. The method according to claim 2, characterized in that... The computation process of any transformer module in the human feature learning model is as follows: in, STA represents the human representation features obtained by the (n-1)th Transformer module. ) represents the overall attention mechanism. LN represents the attentional features obtained from the first global attention mechanism. ) indicates the normalization operation. DS represents the attentional features obtained from the second overall attention mechanism. ) represents a dense mapping. This represents the nth human representational feature obtained at the end.

4. The method according to claim 3, characterized in that... The calculation process of any transformer module in the canine feature learning model is as follows: in, This represents the canine representation features obtained by the (n-1)th Transformer module. This represents the attentional features obtained from the overall attention mechanism. This represents the nth canine appearance feature obtained at the end.

Images