A pet dog emotion recognition method and system based on deep learning

Abstract

The present application relates to the technical field of pet emotion recognition, and particularly relates to a pet dog emotion recognition method and system based on deep learning. The method comprises collecting pet dynamic data, pet physiological data and scene data to obtain a structured data set; the structured data set is labeled by a cross-validation labeling mechanism to obtain a labeled data set; multi-modal features are extracted based on the labeled data set, and the multi-modal features are integrated by a cascade SEblock array to obtain multi-modal fusion features; in the training stage, adversarial sample data generated by a stress scene simulator is injected, and a pet emotion recognition model is trained based on the adversarial sample data and the multi-modal fusion features; and the pet emotion recognition model is used to recognize the emotion of a pet to be recognized to obtain a pet emotion recognition result. The present application improves the data quality and the model robustness of pet emotion recognition, thereby improving the quality of human-pet interaction.

Description

Technical Field

[0001] This invention relates to the field of pet emotion recognition technology, and in particular to a method and system for pet dog emotion recognition based on deep learning. Background Technology

[0002] Traditional methods of pet canine emotion recognition rely primarily on the subjective judgment of owners and the experience of professional veterinarians, lacking objective and quantitative standards and evidence. In recent years, with the rapid development of artificial intelligence technology and the booming pet industry, pet canine emotion recognition has become an emerging research field. Some studies have begun to use computer vision technology to analyze pet facial expressions or use single sensors to collect physiological parameters to infer the emotional state of pets.

[0003] However, existing pet canine emotion recognition technologies have the following shortcomings: First, most methods only focus on single-modal data and ignore the multidimensionality of pet emotional expression, making it difficult to accurately identify complex emotional states; second, existing models perform poorly when dealing with environmental changes and special scenarios (such as veterinary clinics or pet grooming salons, which may trigger stress responses in pets), making it difficult to meet the requirements of model robustness and refined personalization of emotion recognition. Summary of the Invention

[0004] The main objective of this invention is to provide a deep learning-based method for recognizing emotions in pet dogs, aiming to solve the technical problems in the prior art.

[0005] This invention proposes a deep learning-based method for pet dog emotion recognition, comprising:

[0006] Collect pet dynamic data, pet physiological data, and scenario data to obtain a structured dataset;

[0007] A three-level sentiment annotation framework is constructed, and the structured dataset is annotated using a cross-validation annotation mechanism to obtain an annotated dataset;

[0008] Multimodal features are extracted based on the labeled dataset, and multimodal features are integrated through a cascaded SEblock array to obtain multimodal fusion features;

[0009] During the training phase, adversarial sample data generated by a stress scenario simulator is injected, and a deep learning model is trained based on the adversarial sample data and the multimodal fusion features to obtain a pet emotion recognition model.

[0010] The pet emotion recognition model is used to identify the emotions of the pet to be identified, and the pet emotion recognition result is obtained.

[0011] Preferably, the step of annotating the structured dataset using a cross-validation annotation mechanism to obtain an annotated dataset includes:

[0012] The pet's dynamic data is labeled using a labeling tool to obtain basic emotion tags; these basic emotion tags include anxiety, pleasure, and fear.

[0013] The contextual association algorithm is used to associate the contextual scene data with the basic emotion tags to generate context-enhanced tags;

[0014] Based on the physiological parameter mapping model, the correlation coefficient between the pet's physiological data and the emotional state corresponding to the basic emotional tags is calculated to obtain the physiological emotion weight matrix.

[0015] The basic emotion tags, the context enhancement tags, and the physiological emotion weight matrix are cross-validated through a crowdsourcing validation platform. Data with a confidence level below the threshold is removed to obtain the cross-validation results.

[0016] A labeled dataset containing multi-level labels is generated based on the cross-validation results.

[0017] Preferably, the step of extracting multimodal features based on the labeled dataset and integrating the multimodal features through a cascaded SEblock array to obtain multimodal fused features includes:

[0018] Facial region localization was performed based on pet dynamic data using a facial landmark detection network to obtain facial expression features;

[0019] Acoustic feature vectors were obtained by performing audio analysis based on pet dynamic data using Mel-Cepstral time-frequency transform.

[0020] By using long short-term memory networks to model the periodic fluctuations of pet physiological data, physiological state codes are obtained.

[0021] The facial expression features, acoustic feature vectors, and physiological state codes are combined into an initial multimodal feature through a feature concatenation layer.

[0022] The initial multimodal features are optimized by channel weighting using a cascaded SEblock array to generate multimodal fusion features.

[0023] Preferably, the step of optimizing the channel weights of the initial multimodal features using a cascaded SEblock array to generate multimodal fusion features includes:

[0024] The initial multimodal features are globally compressed using the first SEblock module in the cascaded SEblock array to generate the first compressed feature;

[0025] The second compressed feature is obtained by superimposing the variety prior weight on the first compressed feature using the second SEblock module in the cascaded SEblock array;

[0026] The second compressed feature is residually connected by the third SEblock module in the cascaded SEblock array, and the initial multimodal feature is fused with the secondary weighted feature to obtain the preliminary fused feature.

[0027] The initial fusion features are compressed in dimension by a dimensionality reduction fully connected layer to generate multimodal fusion features.

[0028] Preferably, the step of training a deep learning model based on the adversarial example data and the multimodal fusion features to obtain a pet emotion recognition model includes:

[0029] Training input data is generated by an adversarial generative network based on the structured dataset, and the behavioral sequences in the training input data are temporally modeled by a multi-head spatiotemporal Transformer to capture the emotion evolution pattern.

[0030] The contribution ratio of classification loss to regression loss is adjusted through a dynamic weighting strategy;

[0031] The gradient pruning algorithm is used to limit the parameter update magnitude of the deep learning model, prevent overfitting to adversarial examples, and preserve the optimal pet emotion recognition model.

[0032] Preferably, the step of generating training input data using an adversarial generative network and based on the structured dataset includes:

[0033] Enhanced acoustic data is generated by synthesizing the instrument noise spectrum of the structured dataset using a generative adversarial network.

[0034] The pose synthesis algorithm is used to simulate the movement trajectory of pet limbs in pet dynamic data under constraints, and to generate adversarial visual data.

[0035] Simulate the fluctuations in heart rate variability and skin conductivity under fear to generate adversarial physiological data;

[0036] The adversarial visual data, the enhanced acoustic data, and the adversarial physiological data are synchronized into adversarial sample data using a spatiotemporal alignment algorithm.

[0037] The adversarial sample data is mixed with the multimodal fusion features in a proportional manner to generate training input data.

[0038] Preferably, the step of performing emotion recognition on the pet to be identified using the pet emotion recognition model to obtain the pet emotion recognition result includes:

[0039] The multimodal data acquisition system collects real-time data on the pet to be identified, resulting in data to be processed.

[0040] The data to be processed is preprocessed to obtain standardized data to be identified;

[0041] Multimodal feature extraction is performed on the standardized data to be identified to obtain the feature vector to be identified;

[0042] The feature vector to be identified is input into the pet emotion recognition model to obtain the emotion prediction result;

[0043] By using a time-series smoothing filter, short-term abnormal fluctuations in the emotion prediction results are filtered out to obtain a smoothed emotion sequence.

[0044] Based on the smoothed emotion sequence, a pet emotion recognition result containing basic emotion, complex emotion components, and emotion intensity is generated by an emotion state parser.

[0045] This application also provides a deep learning-based pet dog emotion recognition system, including:

[0046] The data acquisition module is used to collect pet dynamic data, pet physiological data, and scenario data to obtain a structured dataset;

[0047] The annotation module is used to construct a three-level sentiment annotation framework and annotate the structured dataset through a cross-validation annotation mechanism to obtain an annotated dataset.

[0048] The extraction module is used to extract multimodal features based on the labeled dataset and integrate the multimodal features through a cascaded SEblock array to obtain multimodal fusion features;

[0049] The training module is used to inject adversarial sample data generated by the stress scenario simulator during the training phase, and to train a deep learning model based on the adversarial sample data and the multimodal fusion features to obtain a pet emotion recognition model.

[0050] The recognition module is configured to use the pet emotion recognition model trained by the training module to perform emotion recognition on the pet to be identified and obtain the pet emotion recognition result.

[0051] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described deep learning-based pet dog emotion recognition method.

[0052] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described deep learning-based pet dog emotion recognition method.

[0053] The beneficial effects of this invention are as follows: This invention proposes a deep learning-based method for pet dog emotion recognition. Through the innovative application of multimodal data fusion and deep learning algorithms, it achieves high-precision recognition of pet emotions, demonstrating significant beneficial effects. First, this invention collects and integrates pet dynamic data, physiological data, and contextual data to construct a comprehensive structured dataset. This breaks through the limitations of traditional single-modal recognition, enabling it to capture the complex expressions of pet emotions from multiple dimensions, significantly improving the comprehensiveness and accuracy of emotion recognition. The system can distinguish subtle emotional changes and transitional states, providing a more comprehensive data foundation for interpreting pet behavior. Second, the three-level emotion annotation framework designed in this invention establishes an objective and systematic emotion annotation system through a progressive mechanism of basic emotion annotation, contextual association enhancement, and physiological parameter mapping. This significantly improves the quality of training data, solves the problems of strong subjectivity and poor consistency in traditional annotation methods, and provides high-quality supervision signals for deep learning models. Third, this invention introduces a cascaded SEblock array to achieve effective fusion of multimodal features. By highlighting key features and suppressing irrelevant information through a channel attention mechanism, this invention significantly enhances feature representation capabilities, enabling the model to more accurately capture the correlation patterns between pet facial expressions, vocal features, and physiological states, thereby improving the accuracy of emotion recognition. Furthermore, by injecting adversarial sample data generated by a stress scenario simulator during training, this invention constructs a recognition model with strong generalization capabilities, ensuring stable recognition performance in complex and ever-changing real-world application scenarios. This effectively solves the problem of decreased recognition accuracy in new environments and special scenarios for traditional models. In specific pet emotion recognition applications, the artificial intelligence algorithm design of this invention is particularly outstanding. The application of a multi-head spatiotemporal Transformer enables the model to effectively capture the temporal evolution of pet emotions, and the design of a dynamic weight loss function balances the dual objectives of emotion classification and physiological parameter regression. The innovative application of the algorithm features in this invention greatly enhances the performance ceiling of the recognition model, enabling the system to identify the complex emotional states of pets in real time and accurately. This provides a scientific basis for pet health monitoring and behavioral intervention, while also building a smoother bridge for emotional communication between owners and pets, improving the human-pet interaction experience. It not only promotes the development of pet welfare science, but also provides technical reference for similar multimodal emotion computing fields, demonstrating the enormous potential and value of artificial intelligence technology in specific application scenarios. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of a method flow according to an embodiment of the present invention.

[0055] Figure 2 This is a schematic diagram of the system structure according to an embodiment of the present invention.

[0056] Figure 3 This is a schematic diagram of the internal structure of a computer device according to an embodiment of this application.

[0057] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0058] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0059] like Figure 1 As shown, this application provides a deep learning-based method for pet dog emotion recognition, including:

[0060] S1. Collect pet dynamic data, pet physiological data, and scenario data to obtain a structured dataset;

[0061] S2. Construct a three-level sentiment annotation framework, and use a cross-validation annotation mechanism to annotate the structured dataset to obtain an annotated dataset;

[0062] S3. Extract multimodal features based on the labeled dataset, and integrate the multimodal features through a cascaded SEblock array to obtain multimodal fusion features;

[0063] S4. During the training phase, adversarial sample data generated by the stress scenario simulator is injected. Based on the adversarial sample data and the multimodal fusion features, a deep learning model is trained to obtain a pet emotion recognition model.

[0064] S5. The pet emotion recognition model is used to perform emotion recognition on the pet to be identified, and the pet emotion recognition result is obtained.

[0065] As described in steps S1-S5 above, this invention collects and integrates pet dynamic data, physiological data, and contextual data to construct a comprehensive structured dataset. This overcomes the limitations of traditional single-modal recognition, enabling the capture of complex expressions of pet emotions from multiple dimensions. It significantly improves the comprehensiveness and accuracy of emotion recognition, allowing the system to distinguish subtle emotional changes and transitional states, providing a more comprehensive data foundation for interpreting pet behavior. Secondly, the three-level emotion annotation framework designed in this invention establishes an objective and systematic emotion annotation system through a progressive mechanism of basic emotion annotation, contextual association enhancement, and physiological parameter mapping. This significantly improves the quality of training data, solves the problems of strong subjectivity and poor consistency in traditional annotation methods, and provides high-quality supervision signals for deep learning models. Thirdly, this invention introduces a cascaded SEblock array to realize multimodal features. The effective integration of various technologies, including channel attention mechanism to highlight key features and suppress irrelevant information, greatly enhances feature representation capabilities. This enables the model to more accurately capture the correlation patterns between pet facial expressions, vocal features, and physiological states, thus improving the accuracy of emotion recognition. Furthermore, this invention injects adversarial sample data generated by a stress scenario simulator during training, constructing a recognition model with strong generalization capabilities. This ensures stable recognition performance in complex and ever-changing real-world application scenarios, effectively solving the problem of decreased accuracy in new environments and special scenarios for traditional models. In specific pet emotion recognition applications, the artificial intelligence algorithm design of this invention is particularly outstanding. The application of a multi-head spatiotemporal Transformer enables the model to effectively capture the temporal evolution of pet emotions, and the design of a dynamic weight loss function balances the dual objectives of emotion classification and physiological parameter regression. The innovative application of the algorithm features in this invention greatly enhances the performance ceiling of the recognition model, enabling the system to identify the complex emotional states of pets in real time and accurately. This provides a scientific basis for pet health monitoring and behavioral intervention, while also building a smoother bridge for emotional communication between owners and pets, improving the human-pet interaction experience. It not only promotes the development of pet welfare science, but also provides technical reference for similar multimodal emotion computing fields, demonstrating the enormous potential and value of artificial intelligence technology in specific application scenarios.

[0066] In one embodiment, the structured dataset in step S1 specifically includes:

[0067] S11. Pet dynamic data, which includes pet visual data, pet sound data, and pet behavior data;

[0068] S12. Pet physiological data, including pet electrocardiogram data, pet respiratory rate data, and pet skin conductivity data;

[0069] S13. Scene data, which includes light intensity data, environmental noise data, and the number of interactive objects.

[0070] As described in steps S11-S13 above, this invention first comprehensively collects pet dynamic data, pet physiological data, and scene data to construct a complete structured dataset. These data collectively constitute a multidimensional feature expression of the pet's emotional state, providing a rich and comprehensive data foundation for subsequent emotion recognition. Pet visual data is primarily captured using high-resolution cameras, capturing facial expressions, body postures, and other visual information. During the acquisition process, image clarity and a frame rate of at least 30fps are required to ensure the capture of subtle facial expression changes and rapid movement reactions. The collected visual data is stored in the form of RGB image sequences or video streams. Each frame contains key information such as the pet's facial region and body posture. In the data processing stage, these raw image data undergo algorithms such as facial region detection and key point localization to extract facial expression features. For example, when acquiring visual data of a Golden Retriever, the facial key point detection algorithm can locate the spatial coordinates of 16 key points, including its eyes, ears, and mouth. These coordinate data form a description of the pet's facial expressions. The feature vectors are derived from pet sound data, which consists of various sound signals emitted by pets, including barking, whimpering, and panting, recorded by a high-sensitivity microphone array. In this embodiment, the microphone sampling rate is set to 44.1kHz to ensure coverage of the entire spectrum of pet sounds. The initial form of the acquired sound data is a time-domain audio waveform, which needs to be converted into a frequency-domain feature representation using the Mel-Cepstral time-frequency transformation algorithm. The final result is an acoustic feature matrix containing multiple frames, with multi-dimensional MFCC feature coefficients extracted from each frame. These feature data can effectively characterize the pitch, timbre, and other acoustic properties of pet sounds. Pet behavior data is a record of pet movement trajectories, activity patterns, and other behavioral information. It is acquired through an inertial measurement unit (IMU) sensor worn on the pet or through video image processing. The IMU sensor can capture changes in the pet's motion state, and the acquired behavioral data is stored in the form of a time series, recording the changes in the pet's acceleration and angular velocity in three-dimensional space. Finally, a feature vector describing the pet's behavior pattern is obtained.

[0071] Pet electrocardiogram (ECG) signal data is collected through pet ECG monitoring equipment, which can capture detailed features of ECG waveforms. The initial form of this ECG data is a time-domain electrical signal waveform, which needs to be processed by algorithms such as R-wave detection and heart rate variability analysis to extract cardiac activity features such as heart rate and heart rate variability. Changes in cardiac activity features reflect the activation state of the sympathetic nervous system. Pet respiratory rate data is collected through devices such as infrared thermal imaging or breathing belts to record changes in the pet's respiratory rhythm. It is stored in the form of respiratory waveforms, and features such as respiratory rate and depth are calculated by peak detection algorithms. The respiratory rate is calculated as the number of respiratory cycles detected per unit time, while the respiratory depth is represented by the peak-to-valley difference of the respiratory waveform. For example, when a dog is relaxed, its respiratory rate may be 15-20 breaths per minute with a stable breathing depth; while when it feels fear, its respiratory rate may increase to 30-40 breaths per minute, and its breathing becomes shallow and rapid. Pet skin conductivity data is collected by electrodes attached to the pet's paw pads or other low-hair areas. Skin conductivity reflects the activity of skin sweat glands and the activation level of the sympathetic nervous system, and is an important indicator for assessing emotional arousal. The data is stored in time series form, recording changes in skin conductivity levels and skin conductivity responses.

[0072] Contextual data is an objective record of the pet's environment, mainly including light intensity data, environmental noise data, and the number of interacting objects. Light intensity data is collected through a light sensor to measure the brightness level of ambient light and record changes in ambient light intensity. In data processing, this invention considers the diurnal variation of light intensity and calculates characteristics such as the rate of light change. For example, a sudden increase in light intensity from normal indoor levels to strong light conditions may cause a stress response in the pet. Environmental noise data is collected through environmental sound monitoring equipment, recording the sound intensity and spectral distribution in the environment. This invention, in processing environmental noise data... During the data processing, the noise intensity of different frequency bands was analyzed, and the frequency and duration of noise events were calculated. The differences in the sensitivity of pet ears to different frequencies of sound were taken into account. The number of interactive objects refers to the number of people or other animals that interact with the pet. These data were obtained through video image analysis and the changes in the number of interactive objects around the pet were recorded. During the data processing, this invention identifies the identity and location of different interactive objects and calculates the frequency and intensity of interactions. For example, when the number of people around a less social dog increases from 1 to 5, it may be observed that the dog changes from a relaxed state to a tense or alert state.

[0073] In one embodiment, step S2, which involves labeling the structured dataset using a cross-validation annotation mechanism to obtain a labeled dataset, includes:

[0074] S21. The pet dynamic data is labeled using a labeling tool to obtain basic emotion tags; wherein the basic emotion tags include anxiety, pleasure, and fear;

[0075] S22. Associate the scenario data with the basic emotion tags using a context association algorithm to generate context-enhanced tags;

[0076] S23. Based on the physiological parameter mapping model, calculate the correlation coefficient between the pet's physiological data and the emotional state corresponding to the basic emotional tags to obtain the physiological emotion weight matrix.

[0077] S24. Cross-validate the basic emotion tags, the context enhancement tags, and the physiological emotion weight matrix through a crowdsourcing validation platform, remove data with a confidence level lower than the threshold, and obtain the cross-validation results.

[0078] S25. Generate a labeled dataset containing multi-level labels based on the cross-validation results.

[0079] As described in steps S21-S25 above, in this invention, the annotation tool mentioned in step S21 refers to a software platform for annotating video and audio data. This platform supports human annotators in encoding and classifying a pet's facial expressions, body posture, and vocal characteristics. Specifically, during the annotation process, annotators observe the pet's dynamic behavioral data, including changes in facial micro-expressions, ear and tail posture, body posture, and changes in pitch, volume, and timbre in the audio data. Based on these observations, the annotator labels each data segment with a basic emotion category, such as anxiety, pleasure, or fear. For example, when an annotator observes a pet dog with its ears tilted back, eyes wide open, mouth twitching, and emitting a low bark, they will label this emotion category as... Behaviors are labeled as "fear"; when a pet is observed to have a relaxed body posture, wagging tail, relaxed facial expression, and cheerful barking, it is labeled as "pleasure"; when a pet is observed pacing back and forth, licking its body, and emitting short, high-frequency barks, it is labeled as "anxiety". The labeling tool records the labels at each time point, and the labeler also adds a confidence score to the label, indicating the degree of certainty about the label. The context association algorithm in step S22 is used to analyze the relationship between environmental factors and emotional expression. Specifically, firstly, the contextual data (including light intensity data, environmental noise data, and the number of interactive objects) is time-aligned with the basic emotional labels, and then the correlation coefficient between each contextual factor and the emotional label is calculated. The context association algorithm uses a conditional probability distribution model to calculate the probability of a certain emotion occurring under specific contextual conditions. The algorithm's formula is:

[0080] ;

[0081] in, In the context Pets expressing emotions The conditional probability, Expressing emotions to pets Situations that occur The conditional probability, Expressing emotions The prior probability, Describing a scenario The prior probability, through the above formula algorithm, can identify which environmental factors are strong triggers for specific emotions. Based on the above association analysis, the algorithm adds context modifiers to the original basic emotion labels to form context-enhanced labels, such as "light-induced anxiety", "pleasure under multi-person interaction" or "fear caused by sudden noise". The obtained context-enhanced labels not only record the emotion type, but also capture the environmental triggers that cause the emotion, thus providing richer semantic information. The physiological parameter mapping model in step S23 is a multivariate statistical model used to quantify the correspondence between physiological signals and emotional states. Specifically, in step S23, the present invention first extracts time-domain and frequency-domain features from pet electrocardiogram signal data, respiratory rate data and skin conductivity data, and then calculates the correlation strength between these features and each emotion category through regression analysis. Through calculation using the physiological parameter mapping model, the correlation between each physiological parameter and each emotion category is quantified into a correlation coefficient. These coefficients collectively form the physiological emotion weight matrix. The calculated correlation coefficients are then organized into this matrix for subsequent emotion state assessment and validation. In step S24, the crowdsourcing validation platform integrates the expertise of multiple professional animal behaviorists, veterinarians, and trainers to validate the automatically generated labeling results. The cross-validation process in step S24 employs a multi-expert voting mechanism, with each data sample evaluated by at least three independent experts. These experts assess the consistency of the three types of labeling results: whether the basic emotion labels accurately reflect the pet's actual emotional state; whether the association between environmental factors and emotions in the context-enhanced labels is reasonable; and whether the correlation coefficients in the physiological emotion weight matrix are consistent with known animal behavior theories. For each sample, experts assign a confidence score and set a confidence threshold. Samples labeled below the confidence threshold are considered unreliable and will be removed or... The data is marked as needing re-labeling. Step S25 uses a multi-level label structure, which is a hierarchical emotion representation method containing three levels: the first level is the basic emotion category (anxiety, pleasure, fear); the second level is the context-modified compound emotion (such as "light-induced anxiety"); and the third level is a detailed description of the emotional state with intensity quantification and physiological association. The structure of the labeled dataset follows the JSON format, and each sample contains the following fields: sample ID, timestamp, basic emotion label, context-enhanced label, emotion intensity value (a floating-point number between 0 and 1), physiological parameter association strength vector (the association strength between each physiological parameter and the emotion), and confidence score. The generation process of the multi-level labels uses a decision tree algorithm to integrate the cross-validated data into a hierarchical label system. The construction of the decision tree is based on the information gain criterion, prioritizing features with strong discriminative power as splitting nodes, ultimately forming a complete multi-level labeled dataset that includes both discrete emotion categories and continuous emotion intensity and environmental factors.

[0082] In one embodiment, step S3, which involves extracting multimodal features based on the labeled dataset and integrating these features using a cascaded SEblock array to obtain multimodal fused features, includes:

[0083] S31. Facial region localization is performed based on pet dynamic data using a facial landmark detection network to obtain facial expression features;

[0084] S32. Audio analysis based on pet dynamic data is performed using Mel-Cepstral time-frequency transform to obtain acoustic feature vectors;

[0085] S33. Model the periodic fluctuations of pet physiological data using a long short-term memory network to obtain physiological state codes;

[0086] S34. The facial expression features, the acoustic feature vector, and the physiological state encoding are merged into an initial multimodal feature through a feature splicing layer;

[0087] S35. The initial multimodal features are optimized by channel weighting through a cascaded SEblock array to generate multimodal fusion features.

[0088] As described in steps S31-S35 above, in step S31 of this invention, a facial key point detection network is used to detect and locate key feature points of a pet's face. The network input is an image frame sequence of the pet's facial region, and the output is the spatial coordinates of the facial key points. The network first uses cascaded convolutional layers to extract multi-scale features of the image, then generates a facial heatmap through a fully convolutional network, and finally determines the precise location of the key points through coordinate regression. In a specific embodiment, for the pet being detected by this invention, 68 facial key points are detected, including eye contours (12 points), nose contours (9 points), mouth contours (20 points), ear contours (12 points), and facial contours (15 points). For canine pets, 56 facial key points were detected, including eye contours (12 points), nose contours (9 points), mouth contours (15 points), ear contours (10 points), and facial contours (10 points). The detection accuracy of facial key points reached the pixel level, with the average detection error controlled within the range of 2-3 pixels. After obtaining the coordinates of the facial key points, this invention further extracted facial expression features, including geometric features and appearance features. Geometric features mainly describe the spatial relationship between key points, such as the distance between the eyes, the degree of eye opening and closing, the degree of mouth opening and closing, and the ear tilt angle. Appearance features were extracted using texture descriptors such as local binary patterns and histograms of orientation gradients to capture subtle changes in facial texture. The geometric features and appearance features were concatenated into a high-dimensional vector image. The quantity constitutes complete facial expression features; the Mel-Cepstral time-frequency transform described in step S32 of this invention is a technique specifically for sound signal processing. It combines the advantages of Mel frequency scale and cepstral analysis, and can effectively extract the spectral features of sound signals. The processing flow first preprocesses the original audio signal, including noise reduction, framing, and windowing. Each frame of the preprocessed signal is sent to the short-time Fourier transform module to calculate its power spectrum. Then, the power spectrum is transformed through the Mel filter bank to convert the linear frequency scale into the Mel frequency scale, which is more in line with human auditory perception. Next, the logarithm of the Mel spectrum is taken, and a discrete cosine transform is performed to obtain the Mel frequency cepstral coefficients (MFCC), the specific expression of which is:

[0089] ;

[0090] in, Indicates the first MFCC coefficients, Indicates the first The output of the Mel filter, Index for Mel filter, Given the total number of Mel filters, this embodiment selects the first 13 MFCC coefficients and calculates their first and second order difference coefficients to form a 39-dimensional basic feature vector. For pet sound analysis, acoustic features including fundamental frequency, harmonic-to-noise ratio, jitter rate, and flicker rate are also calculated, which can reflect the pitch, clarity, and stability of the pet's sound. Finally, all these acoustic features are combined into a complete acoustic feature vector for subsequent emotion analysis. In step S33, this invention uses a Long Short-Term Memory (LSTM) network to capture long-term dependencies in pet physiological data sequences. For pet physiological data modeling, the input to the LSTM is the time series of pet electrocardiogram signal data, respiratory rate data, and skin conductivity data. The LSTM network in this embodiment adopts a bidirectional structure, considering both past and future contextual information. The network contains two layers of LST. M, with 128 hidden units per layer, and an input sequence length of 60 seconds (physiological data collected at a sampling rate of 20Hz); the physiological data is first normalized and then divided into fixed-length sequences according to time windows. The LSTM network learns to capture the temporal dynamic characteristics of physiological signals, such as heart rate variability patterns, respiratory rate trends, and skin conductivity fluctuations. The final hidden state of the network is used as the physiological state encoding, which is a 256-dimensional vector (concatenated from the hidden states of the forward and backward LSTMs), containing a compact representation of the pet's physiological state. In step S34, this invention merges facial expression features, acoustic feature vectors, and physiological state encoding into initial multimodal features through a feature concatenation layer. The feature concatenation layer is a simple and effective multimodal fusion method that directly concatenates feature vectors from different modalities along the feature dimension. The concatenation operation can be represented as:

[0091] ;

[0092] in, This represents the initial multimodal features after concatenation, with dimensions of [dimensionality missing]. , For facial expression features, For acoustic feature vector dimensions, Encoding dimensions for physiological states Indicates facial expression features; Represents the acoustic eigenvector. Represents physiological state encoding. This represents a vector concatenation operation. In a specific line of sight, the facial expression feature dimension is 512, the acoustic feature vector dimension is 128, and the physiological state encoding dimension is 256. The initial multimodal feature dimension after concatenation is 896. Since the scales of different modal features may be inconsistent in direct concatenation, the contribution of different modalities should be different, and the interaction between modalities is not modeled. Therefore, before concatenation, this invention inputs each modal feature into a linear projection layer, maps them to the same feature space, and performs L2 normalization to ensure that the scales of different modal features are consistent. In step S35, this invention optimizes the channel weights of the initial multimodal features through a cascaded SEblock array to generate multimodal fusion features. The SEblock (Squeeze-and-ExcitationBlock) is a channel attention mechanism used to adaptively calibrate the importance of channel features. The cascaded SEblock array set in this invention combines multiple SEblock modules in series to form a deep processing pipeline, which can gradually refine the channel weights to adapt to different levels of feature representation.

[0093] In one embodiment, step S35, which optimizes the channel weights of the initial multimodal features using a cascaded SEblock array to generate multimodal fusion features, includes:

[0094] S351. The initial multimodal features are globally compressed using the first SEblock module in the cascaded SEblock array to generate the first compressed features.

[0095] S352. The first compressed feature is superimposed with the variety prior weight by the second SEblock module in the cascaded SEblock array to obtain the second compressed feature;

[0096] S353. The second compressed feature is residually connected through the third SEblock module in the cascaded SEblock array, and the initial multimodal feature is fused with the secondary weighted feature to obtain the preliminary fused feature.

[0097] S354. The initial fusion features are compressed in dimension by a dimension reduction fully connected layer to generate multimodal fusion features.

[0098] As described in steps S351-S354 above, in step S351, the workflow of the first SEblock module includes two stages: compression (Squeeze) and excitation. In the compression stage, global average pooling is performed on each channel feature to compress the spatial dimension into a single value, obtaining a channel descriptor. In the excitation stage, the nonlinear relationship between channels is learned through a two-layer fully connected network to generate channel weights. Finally, the channel weights are applied to the original features to obtain weighted features. The first SEblock module processes the entire initial multimodal features without distinguishing the modality source. Its main purpose is to suppress noisy channels and enhance channels with high information content. In step S352, the second SEblock module introduces breed prior knowledge into its basic structure. Different breeds of pets differ in facial expressions, vocal features, and physiological reactions, and these differences affect the pattern of emotional expression. For example, the facial expressions of flat-faced breeds (such as Pugs) are relatively difficult to identify, while the ear movements of erect-eared breeds (such as German Shepherds) are more expressive of emotions. To incorporate this prior knowledge, the second SEblock module introduces a breed encoding vector. This is then concatenated with the channel descriptor of the first compressed feature. Next, channel weights are calculated based on the concatenated features. Finally, the variety-aware channel weights are applied to the first compressed feature to obtain the second compressed feature. The specific expression is as follows:

[0099] ;

[0100] in, To obtain the second compression feature, For channel dimension, It is the sigmoid activation function. It is the ReLU activation function. This is the weight matrix of the first fully connected layer. This is the weight matrix of the second fully connected layer. For the first Descriptors for each channel, The first compression feature; variety coding vector The data is obtained through a breed embedding table, which contains codes for 50 common pet breeds. The introduced breed code vectors enable the model to adjust the importance of feature channels for different pet breeds, improving the model's adaptability. In step S353, the third SEblock module introduces a residual connection mechanism to solve the gradient vanishing problem in deep feature extraction while preserving the original feature information. The residual connection is implemented by element-wise addition of the second compressed feature and the initial multimodal feature. Then, the residual feature is further optimized for channel weights through the third SEblock. The significance of step S353 is to fuse the results of multiple rounds of channel weight adjustment with the original features, which preserves the original information and incorporates multi-level channel importance evaluation. In step S354, the initial fused features are compressed in dimension through a dimensionality reduction fully connected layer to generate multimodal fused features. The initial fused features have high dimensionality and redundant information, so dimensionality compression is required. The dimensionality reduction fully connected layer is a linear transformation with a weight matrix and a bias vector, ultimately obtaining the multimodal fused features. Specifically, the initial dimensionality of the fused features is 896, which is reduced to 256 after passing through a dimensionality reduction fully connected layer. Dimensionality reduction not only reduces computational complexity, but also integrates multimodal information by learning the weight matrix, generating a more compact and information-rich representation.

[0101] In one embodiment, step S4, which involves training a deep learning model based on the adversarial example data and the multimodal fusion features to obtain a pet emotion recognition model, includes:

[0102] S41. Generate training input data based on the structured dataset using an adversarial generative network;

[0103] S42. Use a multi-head spatiotemporal Transformer to perform temporal modeling on the behavioral sequences in the training input data to capture the emotion evolution pattern;

[0104] S43. Adjust the contribution ratio of classification loss and regression loss through a dynamic weighting strategy;

[0105] S44. Limit the parameter update magnitude of the deep learning model through gradient pruning algorithm to prevent overfitting of adversarial examples and preserve the optimal pet emotion recognition model.

[0106] As described in steps S41-S44 above, in step S41 of this invention, the Generative Adversarial Network (GAN) is a deep learning architecture composed of a generator and a discriminator. The generator is responsible for generating realistic samples, while the discriminator is responsible for distinguishing between real samples and generated samples. In the pet emotion recognition task, the GAN first learns the data distribution characteristics in the structured dataset, and then generates samples that are similar to real data but contain specific variations. The obtained samples can simulate scenarios that may occur in reality but are not fully expressed in the original dataset. The multi-head spatiotemporal Transformer described in step S42 of this invention is an extension of the standard Transformer architecture, used to process sequence data with spatiotemporal relationships. In pet emotion recognition tasks, a pet's emotional state is not static but a dynamic process that changes over time. Therefore, it is necessary to perform temporal modeling on the pet's behavior sequence. The workflow of the multi-head spatiotemporal Transformer includes: first, dividing the training input data into fixed-length time segments, each segment containing multimodal features; then, adding temporal information to each time step through position encoding; next, using a multi-head self-attention mechanism to calculate the association weights between different time steps to capture long-distance temporal dependencies; and finally, integrating contextual information through a feedforward neural network to generate a feature representation containing temporal semantics. The multi-head mechanism of the multi-head spatiotemporal Transformer allows the model to simultaneously focus on information from multiple subspaces, thereby capturing different types of temporal patterns, such as short-term emotional fluctuations, medium-term emotional trends, and long-term emotional baseline changes. In step S43 of this invention, in the pet emotion recognition task, the model needs to simultaneously complete two tasks: discrete emotion classification (e.g., determining whether it is anxiety, pleasure, or fear) and physiological parameter regression prediction (e.g., predicting heart rate, respiratory rate, etc.). Therefore, the total loss function includes two parts: classification loss and regression loss. The dynamic weight strategy is a method that dynamically adjusts the weights of these two losses during the training process. Its core idea is to focus more on basic classification ability in the early stages of training, and gradually increase the weight of regression loss as training progresses, thereby improving the model's prediction accuracy of physiological parameters. Through the dynamic adjustment strategy, the model can gradually balance classification ability and regression ability during training, ultimately achieving accurate prediction of emotion labels and physiological parameters. The expression for the loss function of the pet emotion recognition model is as follows:

[0107] ;

[0108] ;

[0109] ;

[0110] ;

[0111] in, For the total loss function, For dynamic weights, For discrete emotion classification loss, For physiological parameter regression loss, The initial classification loss weights, To minimize the classification loss weights, This is the weight decay coefficient. For training rounds, For sample index value, The total number of samples, For the emotion category index value, The total number of emotion categories. The smoothed label, For the first Each sample belongs to category The predicted probability, For physiological parameter index values, This represents the total number of physiological parameters. For the first The weight parameters corresponding to each physiological parameter The model predicts the first The first sample The parameter values ​​of each physiological parameter, For the first The first sample The invention specifies the labeled values ​​of several physiological parameters. In step S44, the gradient pruning algorithm is used to limit the parameter update magnitude of the deep learning model, prevent overfitting by adversarial examples, and preserve the optimal pet emotion recognition model. Gradient pruning is a regularization technique to prevent gradient explosion and model overfitting, and it is particularly suitable for the training scenario of this invention that includes adversarial examples. In the gradient pruning algorithm, when the norm of the calculated gradient vector exceeds a preset threshold, the gradient vector is reduced proportionally so that its norm does not exceed the threshold. Through gradient pruning, the update magnitude of the model parameters is limited to a reasonable range, avoiding over-adjustment caused by extreme features in adversarial examples, thereby improving the model's generalization ability and adaptability to new data. During the training process, the model performance is evaluated on the validation set periodically, and the model parameters with the best validation performance are saved, ultimately obtaining the optimal pet emotion recognition model.

[0112] In one embodiment, step S41, which generates training input data based on the structured dataset using an adversarial generative network, includes:

[0113] S411. Perform instrument noise spectrum synthesis on the structured dataset using an adversarial generative network to generate enhanced acoustic data;

[0114] S412. Simulate the pet limb movement trajectory in the pet dynamic data under constrained conditions using a pose synthesis algorithm to generate adversarial visual data.

[0115] S413. Simulate the fluctuations in heart rate variability and skin conductivity under fear to generate adversarial physiological data;

[0116] S414. Synchronize the adversarial visual data, the enhanced acoustic data, and the adversarial physiological data into adversarial sample data using a spatiotemporal alignment algorithm;

[0117] S415. Mix the adversarial sample data with the multimodal fusion features in a proportional manner to generate training input data.

[0118] As described in steps S411-S415 above, in step S411 of this invention, the instrument noise spectrum synthesis refers to the process of simulating the noise generated by various instruments and equipment and superimposing it onto the original pet sound data according to a specific spectral distribution. Specifically, firstly, a short-time Fourier transform is performed on the original pet sound data to obtain a sound spectrum map in the time-frequency domain; then, a generative adversarial network is used to generate spectral features of simulated instrument noise, such as the hum of medical equipment, interference sounds of electronic devices, or mechanical noise in the environment; finally, the generated noise spectrum is weighted and mixed with the original sound spectrum, and converted back to a time-domain signal through an inverse short-time Fourier transform to form enhanced acoustic data. The obtained enhanced acoustic data enables the model to effectively identify the sound signals emitted by the pet in a noisy environment, improving the model's accuracy in emotion recognition under complex acoustic environments. In step S412 of this invention, the skeletal key point sequence is first extracted from the pet dynamic data, and then these key points are geometrically transformed through a Spatial Transformer Network (STN) to generate a representation of the pet's possible action responses under different constraint states. For example, pets restrained in cages or with leashes restricting their movement will have significantly limited limb movements. Pose synthesis algorithms simulate the posture changes pets might make under these restricted conditions, generating corresponding visual data. This synthesized visual data contains subtle visual cues indicating the pet's emotions under constraint, such as changes in ear position, tail wagging patterns, or eye gaze behavior. This helps the model learn to recognize how pets express emotions under different constraints. In step S413, this invention primarily simulates heart rate variability and skin conductance fluctuations under fear, generating adversarial physiological data. Heart rate variability (HRV) is a measure of the change in the time interval between consecutive heartbeats, while skin conductance reflects the level of sympathetic nervous system activity. In a state of fear, a pet's heart rate variability decreases (heart rate interval changes less), while skin conductivity increases. The process of generating adversarial physiological data includes: first, establishing a mathematical model of the physiological response to fear based on medical literature. This model describes the decreasing pattern of heart rate variability and the increasing characteristics of skin conductivity under fear. Then, a random perturbation factor is introduced to simulate individual differences and environmental influences. Finally, time-series physiological signal data is generated, including the changing pattern of RR during electrocardiogram (ECG) data and the dynamic changes of skin conductivity level (SCL). The synthesized physiological data enables the model to better understand the physiological characteristics under fear and improve the accuracy of fear recognition. The spatiotemporal alignment algorithm described in step S414 of this invention is a multimodal data synchronization technology used to ensure that data from different sources and sampling rates are aligned and consistent in the time dimension.Specifically, firstly, each modality of data is timestamped. Then, the optimal alignment path between different modalities is calculated using dynamic time warping. Finally, the temporal resolution of each modality is adjusted using interpolation or downsampling methods to ensure complete alignment in the temporal dimension. Furthermore, the spatiotemporal alignment algorithm considers the causal relationships between different modalities, such as the time delay between a pet's sudden startle and a sudden increase in heart rate observed visually. Through this precise spatiotemporal alignment, the generated adversarial example data accurately reflects the temporal correlation of multimodal signals during emotional changes, thereby helping the model learn cross-modal consistency features of emotional expression. In step S415, this invention employs a proportional mixing strategy, that is, according to a preset mixing ratio, the adversarial example data and multimodal fusion features are weighted and combined. Proportional mixing effectively controls the impact of adversarial examples on model training. By adjusting the ratio, a balance can be achieved between the model's adversarial robustness and basic recognition performance. When the ratio is small, training focuses more on basic recognition capabilities; when the ratio is large, the model's ability to resist adversarial interference is emphasized.

[0119] In one embodiment, step S5, which involves performing emotion recognition on the pet to be identified using the pet emotion recognition model to obtain the pet emotion recognition result, includes:

[0120] S51. The multimodal data acquisition system collects real-time data of the pet to be identified to obtain data to be processed.

[0121] S52. Preprocess the data to be processed to obtain standardized data to be identified;

[0122] S53. Perform multimodal feature extraction on the standardized data to be identified to obtain the feature vector to be identified;

[0123] S54. Input the feature vector to be identified into the pet emotion recognition model to obtain the emotion prediction result;

[0124] S55. Using a time-series smoothing filter, short-term abnormal fluctuations in the emotion prediction results are filtered to obtain a smoothed emotion sequence.

[0125] S56. Based on the smoothed emotion sequence, generate a pet emotion recognition result containing basic emotion, complex emotion components, and emotion intensity through an emotion state parser.

[0126] As described in steps S51-S56 above, in step S51, the multimodal data acquisition system refers to a comprehensive data acquisition platform composed of multiple sensing devices, used to simultaneously acquire multi-dimensional information about a pet, including visual, auditory, behavioral, and physiological data. This system can be the same device used in step S1. The acquired multimodal data is transmitted in real-time to the data processing unit via wireless transmission technology, forming a complete dataset to be processed. In step S52, data preprocessing is the process of converting the raw sensor data into a standard format suitable for deep learning model processing, including data cleaning, format conversion, time alignment, and data standardization. First, data cleaning is performed on each modality of data to remove obvious noise and outliers. For visual data, a median filter algorithm is used to remove noise from video frames, while inter-frame interpolation is performed to supplement any potentially lost frames. For auditory data, a bandpass filter (100Hz-10kHz) is used to filter out ultra-low and ultra-high frequency noise that is inaudible to the human ear, and spectral subtraction is used to suppress background noise. For physiological data, wavelet transform is applied to remove baseline drift and high-frequency interference. Secondly, format conversion is performed to unify the raw data of different formats into a standard data format, such as converting video data into tensor sequences, audio data into Mel spectrograms, and physiological data into normalized time series. Then, a timestamp alignment algorithm is used to align the data of different modalities in the time dimension to ensure the consistency of each modality's data at each point in time. Finally, the data of each modality is standardized to make its distribution characteristics suitable for the processing requirements of deep learning models. In step S53 of this invention, feature extraction is first performed on the visual data. A pre-trained convolutional neural network (CNN) model is used to extract the facial expression features of the pet, and an optical flow algorithm is used to calculate the motion features between consecutive frames to capture the dynamic behavior information of the pet. Further, the facial expression feature extraction process includes three steps: facial region detection, key point localization, and expression encoding. A 2048-dimensional facial expression feature vector is extracted through the first five convolutional layers of the ResNet-50 network. Secondly, feature extraction is performed on the sound data. The Mel frequency cepstral coefficient (MFCC) algorithm is used to extract the acoustic features of the sound, and the energy, fundamental frequency, and other temporal features of the sound are calculated. The MFCC feature extraction process first pre-emphasizes the input audio, then divides it into frames and applies windowing. For each frame, a Fast Fourier Transform (FFT) is performed to obtain the spectrum. This spectrum is then converted to a Mel scale, the logarithm of the Mel spectrum is taken, and finally, a Discrete Cosine Transform (DCT) is performed to obtain the MFCC coefficients. For physiological data, wavelet transform is used to extract the frequency domain features of the electrocardiogram (ECG) signal. Heart rate variability indices (such as SDNN, RMSSD, pNN50, etc.) are calculated, and respiratory rate variation patterns and skin conductance level variation features are extracted. The extracted modal features are combined through feature concatenation to form a multimodal feature vector.To reduce feature dimensionality and improve feature quality, Principal Component Analysis (PCA) is applied for feature dimensionality reduction, retaining principal components that explain 95% of the variance to obtain the final feature vector to be identified. The feature vector to be identified obtained in step S53 is input into the pet emotion recognition model obtained in step S4. The model then outputs the probability distribution of emotion categories and the predicted value of emotion intensity. The emotion prediction results include the probability distribution of basic emotion categories (such as anxiety, pleasure, fear, etc.) and the corresponding emotion intensity values ​​(range 0-1), as well as the predicted values ​​of physiological parameters (such as heart rate, respiratory rate, etc.). The above prediction results constitute the original emotion prediction results. In step S55, the present invention uses a temporal smoothing filter to filter out short-term abnormal fluctuations in the emotion prediction results. After the temporal smoothing filter, the original emotion prediction results are converted into a smoother and more coherent emotion sequence, effectively reducing the interference of random fluctuations in the model prediction process on the emotion recognition results. In step S56, the emotion state parser is a multi-level emotion parsing system used to convert the smooth emotion sequence into pet emotion recognition results with rich semantic information. First, the emotion state parser is based on the VAD (Valence-Arousal-Dominance) emotion model, mapping smooth emotion sequences to a three-dimensional emotion space. Valence represents the positive or negative nature of the emotion, arousal represents the activation level, and dominance represents the sense of control. Through this mapping, the pet's emotional state is represented as a point in the three-dimensional emotion space, facilitating further analysis and quantification. Second, the basic emotion category is determined based on its location in the VAD space. Basic emotions include anxiety, pleasure, and fear, with each basic emotion corresponding to a specific region in the VAD space. Simultaneously, the distance from the emotion vector to each basic emotion prototype vector is calculated to determine the emotion intensity. The closer the distance, the stronger the emotion. For the identification of complex emotions, the emotion state parser uses a fuzzy logic method to calculate the membership degree of the emotion vector to each basic emotion. Basic emotions with a membership degree higher than the threshold constitute complex emotions. The membership degree is calculated using a Gaussian membership function. Finally, the emotion state parser generates a detailed emotion recognition result report, including the dominant basic emotion category, the composition of complex emotions (the proportion of each basic emotion), the emotion intensity value (a scalar in the range of 0-1), and the temporal stability assessment of the emotion state. The multi-level emotion parsing method can provide a richer and more detailed description of the pet's emotional state than simple emotion classification, and can provide valuable emotional reference information.

[0127] Combining the above steps, the entire S5 stage constitutes a complete pet emotion recognition application process. From multimodal data acquisition to the final emotion result output, each step has clear data processing logic and algorithm implementation. Taking a domestic Labrador Retriever as an example, when it is necessary to identify its emotional state during a visit to a veterinary clinic, relevant data is first collected through a multimodal data acquisition system: a camera captures video data of the dog's facial expressions, ear posture, and body movements; a microphone records occasional low whimpers; and a physiological sensor worn on the neck monitors an increase in heart rate from the normal 70 beats / minute to 120 beats / minute, respiratory rate from 18 breaths / minute to 35 breaths / minute, and skin conductivity from the baseline value of 2.1μS to 4.8μS. This raw data is wirelessly transmitted to the processing unit to form the initial dataset to be processed. In the data preprocessing stage, the video data is filtered by median filtering to remove noise caused by unstable lighting, the audio data is filtered by bandpass filtering to remove high-frequency medical equipment noise in the clinic environment, and the physiological data is processed by wavelet transform to remove baseline drift caused by the dog's movement. Subsequently, the data from different modalities were precisely aligned using timestamps and then Z-score standardized to ensure that the distribution characteristics of each type of data met the model input requirements. For example, after standardization, the raw heart rate data of 120 beats / minute was converted into a standardized value of 1.83 (assuming a mean heart rate of 70 and a standard deviation of 27.3). In the feature extraction stage, significant facial expression features were extracted from the video data, such as slightly squinted eyes (eye opening reduced by 20%), ear pressure (ear angle change of 32 degrees), and tightness of the corners of the mouth. 13-dimensional MFCC features were extracted from the sound data, showing characteristics of enhanced low-frequency energy and decreased fundamental frequency. Features such as significantly reduced heart rate variability (SDNN value decreased from 35ms to 12ms) and increased rapid response components of skin conductance were extracted from the physiological data. After feature cascading and PCA dimensionality reduction, the multimodal features formed a 128-dimensional feature vector to be identified. These feature vectors were input into a pre-trained pet emotion recognition model. The model output the corresponding emotion prediction results through multi-level feature fusion and temporal modeling: the probability of anxiety was 0.68, the probability of fear was 0.25, the probability of pleasure was 0.07, and the emotion intensity was 0.75 (range 0-1). Because dogs' emotions fluctuate during treatment, the prediction results may show short-term abnormal fluctuations at certain moments, such as a sudden prediction of a pleasant mood (probability 0.85) lasting for 2 seconds, which is obviously inconsistent with the overall situation.Using a temporal smoothing filter (with a smoothing factor α = 0.15), these short-term abnormal fluctuations were effectively smoothed, resulting in a more coherent emotional sequence. The corrected emotional probabilities became: anxiety 0.72, fear 0.21, and pleasure 0.07, with emotional intensity fluctuating within the range of 0.7-0.8. Finally, based on the smoothed emotional sequence, the emotional state parser mapped the emotional state to the VAD space, obtaining an effectiveness value of -0.6 (negative emotion), an arousal value of 0.75 (high activation), and a dominance value of -0.3 (moderate to low sense of control). Based on these parameters, the parser determined that the dog's dominant basic emotion was anxiety, while also containing a component of fear, constituting a complex emotion. Specifically, anxiety accounted for 75%, fear accounted for 25%, and the emotional intensity was 0.73 (moderate to high intensity). The final pet emotion recognition results showed that the Labrador was in a state of moderate to high-intensity anxiety-fear complex emotion in the veterinary clinic environment. The main manifestations were multimodal features such as facial tension, ear retraction, low whimpering, increased heart rate, and enhanced skin conductance. The emotion recognition results provided veterinarians and pet owners with objective and accurate emotional reference information, which helped to adjust treatment methods and improve the pet's medical experience.

[0128] like Figure 2 As shown, the present invention also provides a deep learning-based pet dog emotion recognition system, comprising:

[0129] The data acquisition module is used to collect pet dynamic data, pet physiological data, and scenario data to obtain a structured dataset;

[0130] The annotation module is used to construct a three-level sentiment annotation framework and annotate the structured dataset through a cross-validation annotation mechanism to obtain an annotated dataset.

[0131] The extraction module is used to extract multimodal features based on the labeled dataset and integrate the multimodal features through a cascaded SEblock array to obtain multimodal fusion features;

[0132] The training module is used to inject adversarial sample data generated by the stress scenario simulator during the training phase, and to train a deep learning model based on the adversarial sample data and the multimodal fusion features to obtain a pet emotion recognition model.

[0133] The recognition module is configured to use the pet emotion recognition model trained by the training module to perform emotion recognition on the pet to be identified and obtain the pet emotion recognition result.

[0134] This invention overcomes the limitations of a single modality by simultaneously acquiring dynamic visual / sound data, physiological signals, and environmental scene information (such as electrocardiogram, noise intensity, etc.). Compared with traditional methods that rely solely on audio or images (which suffer from environmental interference or individual differences), this invention optimizes channel weights through a cascaded SEblock array, effectively fusing facial expressions, acoustic features, and physiological state encoding. The triple data collaboration of this invention further enhances the integrity of emotional representation.

[0135] This invention employs a cross-validation mechanism combining basic emotion labels, context-enhanced labels, and a physiological weight matrix, addressing the inherent subjectivity of traditional manual labeling. Specifically, this invention utilizes a crowdsourcing platform to eliminate low-confidence data (such as the aforementioned insufficient labeled samples) and incorporates breed-specific prior weight adjustments, effectively mitigating interference from differences in emotional expression among different pet breeds. Compared to existing single-labeling methods (such as classification based solely on sound), this framework makes the labeling system more closely aligned with real-world contexts.

[0136] By injecting adversarial examples such as the noise spectrum of equipment and simulated limb movement trajectories during the training phase, and preventing overfitting through gradient clipping, the generalization ability of the model in complex environments (such as high noise and occlusion scenarios) is significantly improved. Adversarial training not only improves the verification accuracy, but also enhances the model's adaptability to sudden stress scenarios by realizing the synchronous generation of multimodal adversarial data through the spatiotemporal alignment algorithm.

[0137] A dynamic weighting strategy combining classification and regression losses is employed, initially focusing on discrete emotion classification and gradually increasing the supervision intensity of physiological parameter regression. Compared to fixed-weight models (such as CNN models), this adaptive mechanism improves validation accuracy while ensuring a strong correlation between physiological indicators (such as heart rate variability) and emotional state.

[0138] By modeling behavioral sequences using a multi-head spatiotemporal Transformer and combining it with a post-temporal smoothing filter, this method effectively identifies continuous changes in emotional states (such as the transition from fear to anxiety). Compared to static acoustic feature analysis, this method can capture dynamic features such as tail wagging frequency and pupil constriction, significantly improving the accuracy of identifying complex emotions (such as "fear accompanied by aggressive tendencies").

[0139] This invention can quantify the emotional state of pets, providing a quantitative basis for pet health monitoring (such as stress response early warning) and behavior correction training, which can reduce the misdiagnosis rate of pet medical care. At the same time, through emotional intensity analysis, it can provide data support for the development of personalized interaction plans.

[0140] It should be noted that each module and unit in the deep learning-based pet dog emotion recognition system corresponds one-to-one with the steps in the deep learning-based pet dog emotion recognition method.

[0141] like Figure 3 As shown, this application also provides a computer device, which can be a server, and its internal structure can be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores all the data required for the process of the deep learning-based pet dog emotion recognition method. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the deep learning-based pet dog emotion recognition method.

[0142] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment on which the present application is applied.

[0143] An embodiment of this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any of the above-described deep learning-based methods for pet dog emotion recognition.

[0144] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in this application and in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0145] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0146] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A deep learning-based pet dog emotion recognition method, characterized by, The method comprises the following steps: Collecting pet dynamic data, pet physiological data, and scene data to obtain a structured data set; Constructing a three-level emotion labeling framework, labeling the pet dynamic data through a labeling tool to obtain basic emotion labels, wherein the basic emotion labels include anxiety, joy, and fear; Associating the scene data with the basic emotion labels through a context association algorithm to generate context-enhanced labels; Calculating the correlation coefficient of the pet physiological data and the corresponding emotion state of the basic emotion labels based on a physiological parameter mapping model to obtain a physiological emotion weight matrix; Cross-verifying the basic emotion labels, the context-enhanced labels, and the physiological emotion weight matrix through a crowdsourcing verification platform, removing data with a labeling confidence lower than a threshold, and obtaining a cross-verification result; Generating a labeling data set containing multi-level labels according to the cross-verification result; Positioning a facial region based on pet dynamic data through a facial key point detection network to obtain facial expression features; Performing audio analysis based on pet dynamic data through Mel-Cepstral time-frequency transformation to obtain an acoustic feature vector; Modeling periodic fluctuations of pet physiological data through a long short-term memory network to obtain physiological state encoding; Merging the facial expression features, the acoustic feature vector, and the physiological state encoding into initial multi-modal features through a feature splicing layer; Performing global channel compression on the initial multi-modal features through a first SEblock module in a cascaded SEblock array to generate first compressed features; The first compressed feature is superimposed with a variety prior weight through a second SEblock module in the cascade SEblock array, the second SEblock module introduces a variety code vector And the channel descriptor of the first compressed feature is spliced, then, the channel weight is calculated based on the spliced feature, finally, the variety-aware channel weight is applied to the first compressed feature to obtain the second compressed feature, and the specific expression is: ； wherein, is a second compressed feature obtained, is a channel dimension, is a sigmoid activation function, is a ReLU activation function, is a first fully connected layer weight matrix, is a second fully connected layer weight matrix, is a descriptor of a first channel, is a descriptor of an nth channel, is a first compressed feature; Performing residual connection on the second compressed features through a third SEblock module in the cascaded SEblock array to fuse the initial multi-modal features and secondary weighted features, and obtaining preliminary fusion features; Performing dimension compression on the preliminary fusion features through a dimension reduction fully connected layer to generate multi-modal fusion features; Injecting adversarial sample data generated by a stress scene simulator in a training phase, training a deep learning model based on the adversarial sample data and the multi-modal fusion features, and obtaining a pet emotion recognition model; Performing emotion recognition on a pet to be recognized through the pet emotion recognition model to obtain a pet emotion recognition result. 2.The deep learning-based pet dog emotion recognition method of claim 1, wherein, The step of training the deep learning model based on the adversarial sample data and the multi-modal fusion features to obtain the pet emotion recognition model comprises the following steps: Generating training input data based on the structured data set through a generative adversarial network, and modeling the behavior sequence in the training input data through a multi-head spatio-temporal Transformer to capture the emotion evolution pattern; Adjusting the contribution proportion of classification loss and regression loss through a dynamic weight strategy; Limiting the parameter update amplitude of the deep learning model through a gradient clipping algorithm to prevent overfitting of the adversarial sample, and saving the optimal pet emotion recognition model. 3.The deep learning-based pet dog emotion recognition method of claim 2, wherein, The step of generating training input data based on the structured data set through a generative adversarial network comprises the following steps: Performing instrument noise spectrum synthesis on the structured data set through a generative adversarial network to generate enhanced acoustic data; The pet limb motion trajectory in the pet dynamic data under the constraint state is simulated through a pose synthesis algorithm to generate adversarial visual data; The heart rate variability and skin conductivity fluctuation under the fear emotion are simulated to generate adversarial physiological data; The adversarial visual data, the enhanced acoustic data and the adversarial physiological data are synchronized into adversarial sample data through a space-time alignment algorithm; The adversarial sample data and the multi-modal fusion features are mixed in proportion to generate training input data. 4.The deep learning-based pet dog emotion recognition method of claim 1, wherein, The pet emotion recognition model is used to recognize the emotion of the pet to be identified to obtain a pet emotion recognition result. Real-time data of the pet to be identified is collected through a multi-modal data acquisition system to obtain processing data; The processing data is preprocessed to obtain standardized data to be identified; Multi-modal feature extraction is performed on the standardized data to be identified to obtain a feature vector to be identified; The feature vector to be identified is input into the pet emotion recognition model to obtain an emotion prediction result; Short-term abnormal fluctuations in the emotion prediction result are filtered through a time series smoothing filter to obtain a smoothed emotion sequence; According to the smoothed emotion sequence, an emotion state analyzer is used to generate a pet emotion recognition result including a basic emotion, a composite emotion composition and an emotion intensity.

5. A deep learning-based pet dog emotion recognition system, characterized by, The system includes: A collection module is configured to collect pet dynamic data, pet physiological data and scene data to obtain a structured data set; A labeling module is configured to construct a three-level emotion labeling framework and label the pet dynamic data through a labeling tool to obtain a basic emotion label; the basic emotion label includes anxiety, joy and fear; A context association algorithm is used to associate the scene data with the basic emotion label to generate a context-enhanced label; A physiological parameter mapping model is used to calculate a correlation coefficient of the pet physiological data and the emotion state corresponding to the basic emotion label to obtain a physiological emotion weight matrix; A crowd verification platform is used to cross-verify the basic emotion label, the context-enhanced label and the physiological emotion weight matrix, remove data with a labeling confidence lower than a threshold, and obtain a cross-verification result; An annotation data set including multi-level labels is generated according to the cross-verification result; A feature extraction module is configured to locate a facial region based on pet dynamic data through a facial key point detection network to obtain facial expression features; Mel-Cepstral time-frequency transformation is used to analyze audio based on pet dynamic data to obtain an acoustic feature vector; A long short-term memory network is used to model periodic fluctuations of pet physiological data to obtain physiological state encoding; A feature splicing layer is used to combine the facial expression features, the acoustic feature vector and the physiological state encoding into initial multi-modal features; A first SEblock module in a cascaded SEblock array is used to perform global channel compression on the initial multi-modal features to generate first compressed features; The first compressed feature is superimposed with a variety prior weight through a second SEblock module in the cascade SEblock array, the second SEblock module introduces a variety code vector And the channel descriptor of the first compressed feature is spliced, then, the channel weight is calculated based on the spliced feature, finally, the variety-aware channel weight is applied to the first compressed feature to obtain the second compressed feature, and the specific expression is: ； wherein, is a second compressed feature obtained, is a channel dimension, is a sigmoid activation function, is a ReLU activation function, is a first fully connected layer weight matrix, is a second fully connected layer weight matrix, is a descriptor of a first channel, is a descriptor of an nth channel, is a first compressed feature; A third SEblock module in the cascaded SEblock array is used to perform residual connection on the second compressed features, fuse the initial multi-modal features and the second weighted features, and obtain preliminary fusion features; The preliminary fusion features are dimensionally compressed by a dimension reduction fully connected layer to generate multi-modal fusion features; The training module is configured to inject the adversarial sample data generated by the stress scene simulator in a training phase, train a deep learning model based on the adversarial sample data and the multi-modal fusion features, and obtain a pet emotion recognition model. The recognition module is configured to use the pet emotion recognition model trained and obtained by the training module to perform emotion recognition on a pet to be recognized, and obtain a pet emotion recognition result. 6.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-5 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 4.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 4.

Images